ThroughHand: 2D Tactile Interaction to Simultaneously Recognize and Touch Multiple Objects

Document Title

ThroughHand: 2D Tactile Interaction to Simultaneously Recognize and Touch Multiple Objects

Document Information

• Subject Area: Human-Computer Interaction, focusing on tactile interaction technologies for visually impaired users
• Keywords: Tactile sensing, dynamic interaction, visual impairment, accessibility technology, 2D tactile display, game design, palm perception, multi-object real-time interaction, human-computer interaction

Research Background and Problem

• Problem or Challenge:
  • Traditional touchscreen interactions are challenging for visually impaired users, especially in applications requiring quick recognition and manipulation of multiple dynamic targets (e.g., sports games).
  • Current systems that substitute vision with auditory or tactile feedback have limitations, such as auditory information overload, imprecise directional localization, and lack of real-time tactile input capabilities.
• Research Significance:
  • Enhancing the ability of visually impaired users in real-time interactions and dynamic tasks could reshape their entertainment experiences and daily human-computer interactions.
  • By developing new forms of tactile interaction, this research aims to address technological gaps in multi-target real-time recognition and manipulation.

Solution

• Proposed Solution:
  • A tactile interaction method called ThroughHand was developed, utilizing tactile feedback on both the palm and the back of the hand to enable visually impaired users to recognize and manipulate multiple objects in a 2D dynamic space in real time.
• Innovations:
  • Delivering tactile output through the palm while using the back of the hand as a touch input space, creating an integrated tactile interaction system.
  • Proposing and validating the feasibility of perceiving multiple object positions through the palm and interacting via the back of the hand.
• Implementation Steps and Key Technologies:
  • Using shape-changing tactile display devices to deliver stimuli on the palm.
  • Employing optical tracking technology to detect touch positions on the back of the hand as input.
  • Enhancing accuracy through personalized calibration, including recording perceivable points and touch error thresholds.

Research Outcomes

• Specific Results:
  • User studies demonstrated that the system could locate stimuli on the palm with an error margin of approximately 13 millimeters and provide effective 2D real-time interaction experiences across various game scenarios.
  • Successfully developed and tested four game applications (Whack-a-Mole, Shooting, Multiplayer Ping Pong, Rhythm Game), validating ThroughHand's practical applications.
• Advantages Compared to Existing Solutions:
  • Capable of simultaneously perceiving multiple dynamic objects and enabling real-time interaction.
  • Reduces auditory information overload and supports more intuitive tactile feedback and input.
• Experimental or Evaluation Results:
  • Accuracy: Average localization error of 13.2 millimeters, with only an 18% delay in touch selection time compared to vision-based systems.
  • Game Performance: Users adapted easily to game operations (e.g., Whack-a-Mole success rates ranged from 67% to 94%), showcasing new possibilities for tactile-based gaming.
• Limitations and Future Directions:
  • The resolution of the tactile display is limited by the two-point discrimination threshold of the palm; improving resolution could reduce the discreteness of moving objects.
  • Some users reported that device adaptability needs improvement, such as limited perception range for smaller hands.
  • Enhancing shape recognition capabilities (e.g., using more distinct tactile patterns) and exploring ways to reduce the time required for personalized calibration.
  • Potential future applications include dynamic graphical user interfaces and web-assisted browsing.

Conclusion

ThroughHand innovatively combines palm perception and back-of-hand input to provide visually impaired users with an effective new form of 2D tactile interaction. The results demonstrate its broad application potential, from gaming to everyday dynamic interactions. However, future research should focus on optimizing resolution and user adaptability while exploring a wider range of practical application scenarios.