Investigating Eyes-away Mid-air Typing in Virtual Reality using Squeeze haptics-based Postural Reinforcement

Title of the Paper

Investigating Eyes-away Mid-air Typing in Virtual Reality using Squeeze Haptics-based Postural Reinforcement

Paper Information

• Research Area: Human-Computer Interaction, Text Input in Virtual Reality
• Keywords: Virtual Reality, Mid-air Typing, Haptic Feedback, Postural Reinforcement, Wrist Squeeze Feedback, Eye Tracking

Research Background and Problem

• Identified Issues or Challenges:

  • In virtual reality, mid-air typing lacks the tactile feedback provided by physical keyboards, making it difficult for users to achieve "eyes-away typing."
  • While previous studies have explored the impact of vibration-based haptic feedback on typing experience, they found no significant improvement in typing speed and accuracy, and did not thoroughly quantify how haptic feedback could potentially reduce visual and cognitive load.

• Significance:

  • Achieving eyes-away typing is critical for making virtual keyboard technology more aligned with human typing habits. This not only enhances user experience but also has the potential to improve productivity.

• Motivation and Related Work:

  • Previous research has shown that vibration-based haptics can reduce users' perceived cognitive load to some extent but failed to improve typing performance.
  • Building on this foundation, this study proposes a novel haptic feedback model (including critical wrist compression feedback) and introduces eye-tracking metrics to more comprehensively quantify its impact.

Solution

• Methods and Approach:

  • Proposed three experimental conditions based on wrist haptic feedback and one control condition (no haptics):
    1. No Haptics (NH)
    2. Keypress Vibration Feedback (VK)
    3. Keypress Vibration + Linear Squeeze Feedback (SVK)
    4. Postural Reinforcement Squeeze Feedback + Keypress Vibration Feedback (SPVK)
  • Utilized "postural reinforcement" wrist squeeze feedback to help users maintain better wrist positioning and posture, potentially reducing visual dependency on the keyboard.

• Innovations:

  • Introduced postural reinforcement haptic feedback to simulate the support provided by physical keyboards for hand posture.
  • Employed objective metrics such as eye-tracking to quantify the potential impact of haptic feedback on users' visual and cognitive behavior.

• Implementation Steps and Core Techniques:

  • Designed wrist-worn devices (Tasbi) with compression and vibration feedback capabilities to create different haptic feedback conditions.
  • Used Quest 2 VR with an embedded Tobii eye-tracking system.
  • Conducted a text input experiment with 48 participants (divided into novice and expert groups).
  • Collected input metrics such as typing speed, error rate, and key collision count, as well as participants' gaze metrics (e.g., time spent looking at the keyboard, frequency of gaze shifts).

Research Findings

• Specific Findings:

  • The SPVK condition (postural reinforcement + vibration haptic feedback) significantly reduced expert users' visual dependency on the keyboard by 44% during typing.
  • The SVK condition showed a significant reduction in the number of non-target key collisions by users.
  • Although typing speed and accuracy did not significantly improve, haptic feedback reduced users' visual load, enabling expert users to approach eyes-away typing behavior.

• Comparison with Existing Solutions:

  • Introduced postural reinforcement haptic feedback, representing a breakthrough compared to previous studies that focused solely on keypress feedback.

• Experimental or Evaluation Results:

  • For eye-tracking metrics, SPVK significantly reduced the time spent looking at the keyboard and demonstrated broader applicability.
  • The tendency to reduce visual attention was more pronounced among expert users.
  • While no significant improvements in typing speed and accuracy were observed, user experience and attention allocation showed notable enhancements.

• Limitations and Future Directions:

  1. Significant differences were observed between novices and experts. Future research could explore how to customize feedback to better suit novice users.
  2. The wrist vibration/compression feedback investigated in this study showed limitations in long-term usage stability. Future work could test the effects of haptic reinforcement over extended periods.
  3. Expanding eye-tracking metrics research could evaluate the applicability of haptic reinforcement in different interaction tasks (e.g., grasping and manipulation tasks).

Summary and Value

This study innovatively introduces postural reinforcement haptic feedback into mid-air typing in virtual reality, demonstrating its potential to significantly improve "eyes-away typing" capabilities for the first time. It also provides a new method for quantifying the effects of haptic reinforcement (e.g., through eye-tracking). This research offers valuable guidance for the development of next-generation virtual interaction designs and haptic interaction devices, highlighting the profound impact of haptic feedback on user cognition and behavioral patterns.