Self-Distillation Based Multi-task Learning Model for Stylus Input Latency Compensation

DOI

Input latency significantly deteriorates the users experience during touchscreen interactions, especially when they engage in precision tasks such as writing or drawing with a stylus. We address this issue by first decomposing it into two constituent tasks: stylus nib future trajectory prediction and predicted trajectory length optimization, facilitating a more thorough investigation into balancing latency compensation and side-effects, and then proposing a novel multi-task learning architecture that integrates the consideration of both tasks, enhances overall performance through alternating-joint training. Additional usage of specific features generated with an active stylus and the adoption of a customized distance error metric are also aimed at accuracy improvement. Experiments reveal that the multi-task learning model gives 0.47, 1.30, and 2.24 pixels of average error in cases of prediction in 6, 14, and 20 ms, and offers better trade-offs between latency reduction and side-effects across a wide range of usage scenarios.