TapGazer: Text Entry with Finger Tapping and Gaze-directed Word Selection

Document Title

TapGazer: Text Entry with Finger Tapping and Gaze-directed Word Selection

Document Information

• Subject Area: Text input technologies in Human-Computer Interaction and Virtual Reality
• Keywords: Input technology, text entry, eye tracking, virtual reality, typing

Research Background and Problem

• Problems and Challenges:

  • Efficient text entry in virtual reality (VR) environments is challenging, as VR headsets prevent users from easily locating traditional physical keyboards.
  • Users often stand or move around, making it difficult to stay close to a keyboard.
  • Many existing solutions require users to learn new skills or layouts, such as non-traditional typing methods, which reduces learning efficiency.

• Importance:

  • Text entry is a fundamental task in computing systems, and efficient text input methods are crucial for productivity.
  • VR scenarios requiring quick text input (e.g., note-taking or messaging) demand a simple and user-friendly solution.

• Related Work and Research Motivation:

  • Traditional VR text input methods include virtual keyboards, portable keyboards, gesture input, and eye-tracking control, but their performance rarely surpasses physical keyboards.
  • TapGazer aims to address issues of portability, visibility, and learning cost in VR scenarios while preserving users' familiarity with traditional QWERTY typing.

Solution

• Method and Design:

  • A text entry system combining finger tapping and gaze-directed word selection—TapGazer—is proposed.
  • Users input text by tapping their fingers without needing to focus on hand or keyboard positions.
  • The system utilizes a QWERTY layout to map multiple letters to each finger and resolves input ambiguities by displaying a list of candidate words for gaze-based selection.

• Innovations:

  • First exploration of combining tapping with gaze-based input to resolve word ambiguities.
  • Provides a flexible and portable text input method in VR scenarios, supporting both standing and seated use.
  • Beginner-friendly and easy to learn, retaining users' QWERTY typing skills.

• Implementation Details:

  • Eye tracking is used for word selection, reducing additional tapping actions; in the absence of eye-tracking devices, extra finger taps can be used for word selection.
  • Multiple candidate word layout designs (e.g., pentagonal layout) are provided to optimize visual search efficiency.
  • The system supports various input devices, such as QWERTY keyboards, Sensel touchpads, and wearable touch gloves.

Research Outcomes

• Specific Results:

  • TapGazer enables novice users to achieve 44.81 words per minute (WPM) in seated VR scenarios, approximately 79.17% of their QWERTY keyboard typing speed.
  • In standing VR scenarios, users with touch gloves reached 45.26 WPM, approximately 71.91% of QWERTY speed.
  • Compared to existing text input technologies, TapGazer demonstrates superior performance and significantly reduces error rates.

• Advantages and Experimental Results:

  • In Word Completion mode, users saved an average of 7% of tapping actions.
  • Compared to physical keyboards, error rates were significantly reduced (from 11.55% to 2.13%-3.64%).
  • Users provided positive experience ratings (SUS usability: ~70 points, NASA-TLX task load: ~48 points).
  • Performance remained consistent across seated and standing postures, with no significant loss.

• Limitations and Future Directions:

  • TapGazer heavily relies on the accuracy of eye tracking, which depends on users' visual input and does not support typing without visual focus.
  • The system lacks automatic error correction functionality; future improvements could integrate advanced prediction algorithms and error correction mechanisms.
  • With the proliferation of AR glasses and advanced finger tracking devices, TapGazer could become a widely adopted portable text input method.

Conclusion

TapGazer combines tapping and eye tracking to achieve an efficient and easy-to-learn text input system suitable for everyday tasks in virtual reality environments. This system demonstrates the potential for flexible applications in non-standard environments and provides a reference for future design optimizations based on gaze and tapping inputs. Future research could further enhance prediction algorithms and user experience design, making it applicable to more scenarios, such as AR applications or everyday mobile use cases.