UniKey: Enabling Surface-Based Typing with Commodity Smartwatches via Cross-Modal Learning

DOI

With the rise of mobile technologies such as augmented and virtual reality (AR/VR) and wearables, as well as smart TVs, the large form factor of traditional keyboards is becoming increasingly impractical. Sensing typing on surfaces offers a potential alternative, but most current solutions rely on either expensive, custom hardware or extensive user bootstrapping and calibration. In this paper, we envision Unikey, an approach to surface-based typing that uses only a commodity smartwatch to detect finger taps on a flat surface. By seamlessly adapting to the user’s existing typing habits on conventional keyboards, Unikey eliminates the need for both specialized equipment and burdensome sensor data collection, reducing overhead for users. To demonstrate the feasibility of our approach, we implement a proof-of-concept and evaluate our technique with comprehensive real-world experiments under varying conditions. Participants were invited to type while wearing smartwatches, resulting in over 2,700 minutes of recorded typing. Our experiments show that Unikey can achieve an equivalent average top-5 word error rate of 6.45%, indicating a potential solution simple, everyday text-entry tasks.