ReachPad: Interacting with Multiple Virtual Screens using a Single Physical Pad through Haptic Retargeting

Research Background and Issues

• Identified Problems or Challenges
  With the development of virtual reality (VR) technology, two-dimensional user interfaces have been extended into three-dimensional spaces. However, the lack of tactile feedback has emerged as a significant issue, negatively impacting user experience and precise operations, particularly in cross-screen selection and interaction tasks. Existing solutions often require specialized haptic devices or highly customized hardware, and there is limited research on how to interact with multiple virtual screens using a single physical proxy.

• Significance
  When users operate multiple virtual screens in a 3D space, tactile feedback not only enhances immersion and satisfaction but also significantly reduces interaction errors, improves control accuracy, and increases task efficiency. However, current technologies are not easily integrated with multiple virtual screens, highlighting the importance of exploring a low-cost, easily implementable solution.

• Research Motivation and Related Work
  To address the above challenges, this paper draws on prior research on virtual screen interaction technologies and haptic redirection techniques. By combining the advantages of interaction and tactile feedback, the authors propose a multi-screen haptic redirection method based on a single physical board and conduct detailed experiments to validate its performance and design space.

Solution

• Proposed Approach
  The authors propose "ReachPad," an interaction method that maps multiple virtual screens onto a single physical board. Users interact with a fixed physical board and perceive multiple virtual screens through haptic redirection techniques, enabling cross-screen operations without requiring multiple hardware devices or complex environments.

• Innovations

  1. Utilizing haptic redirection to dynamically alter the mapping relationship between the virtual hand and the physical board, thereby expanding the functionality of a single physical board.
  2. Employing eye-tracking technology to enable natural and seamless switching between multiple virtual screens.
  3. Proposing three cross-screen drag-and-drop methods and validating their performance through experiments, providing design references for future multi-screen interactions.
  4. Extending the feasibility of low-cost tactile proxies to support complex multi-screen tasks.

• Implementation Steps and Key Technologies

  1. Using haptic redirection techniques to dynamically redirect the user's virtual hand to virtual screens while maintaining synchronization between the physical hand and the actual input board.
  2. Leveraging eye-tracking technology to switch the virtual screen mapped to the physical board in real-time, ensuring smooth and natural screen transitions.
  3. Designing and testing three cross-screen drag-and-drop methods (corner method, long-press method, and edge method) to optimize user experience.
  4. Conducting experimental comparisons between "ReachPad" and traditional interaction devices (e.g., controllers and mid-air interactions) in practical application scenarios.

Research Outcomes

• Specific Results

  1. Optimized Design: Experiments identified the optimal virtual screen size (46-degree field of view) and position (preferably lower-right).
  2. Cross-Screen Interaction Innovation: The proposed edge method performed best in cross-screen drag-and-drop tasks, balancing speed, path efficiency, and low error rates.
  3. Practical Validation: Compared to existing methods (mid-air interaction and controllers), ReachPad significantly outperformed in terms of immersion and enjoyment while maintaining a low task workload.

• Advantages Compared to Existing Solutions

  • Compared to controller-based interaction, ReachPad offers more precise drag-and-drop operations, making it suitable for continuous and complex tasks.
  • Compared to mid-air interaction, tactile feedback reduces hand fatigue, enhances natural interaction experiences, and lowers error rates.
  • It does not require additional physical interfaces, offering high cost-effectiveness and practical usability.

• Experimental or Evaluation Results

  1. In real-world application scenarios (e.g., drawing, nightclub control), ReachPad scored significantly higher in immersion and enjoyment compared to other methods.
  2. Experimental data showed that the edge method achieved the highest efficiency in cross-screen drag-and-drop tasks, with the lowest error rate and highest applicability.
  3. NASA-TLX evaluations indicated that ReachPad's user task workload was lower than mid-air interaction and comparable to controller-based methods.

• Limitations and Future Directions

  1. Hardware Limitations: The current implementation requires an external high-precision hand-tracking system (e.g., Optitrack). Future work may explore compatibility with embedded tracking solutions.
  2. Design Extensions: While single-finger operations have proven effective, further research is needed to explore the effects of multi-finger and bimanual interactions using haptic redirection.
  3. Precision Challenges: Depth-axis operations and haptic redirection in non-offline environments need further optimization to improve performance in complex task scenarios.
  4. Application Expansion: Investigate how ReachPad's design can be extended to real-world office environments and its potential to enhance workflow efficiency.

This paper provides a low-cost and innovative solution for multi-screen interaction in VR, overcoming the physical limitations of existing methods and demonstrating exceptional performance across various tasks. Future research could expand on more complex scenarios and broader user needs to further advance the practical application of multi-screen VR interaction technologies.