Stereoscopic Viewing and Monoscopic Touching: Selecting Distant Objects in VR Through a Mobile Device

Title of the Paper

Stereoscopic Viewing and Monoscopic Touching: Selecting Distant Objects in VR Through a Mobile Device

Paper Information

• Field of Study: Virtual Reality (VR) Interaction Technology
• Keywords: Virtual Reality, Smartphone, Transparency, Distant Selection, Binocular Parallax

Research Background and Problem

• Identified Problems or Challenges:

  • When using distant selection techniques in VR, binocular parallax makes it difficult for users to accurately select target objects.
  • Many current VR interaction techniques based on mobile devices fail to effectively address the challenges of distant interaction, such as accuracy and intuitiveness in object selection.

• Significance:

  • As VR technology becomes more widespread, everyday devices like smartphones could become essential tools for interacting with virtual environments. Combining VR's immersive visual output with the tactile input of smartphones could enhance the efficiency and experience of human-virtual environment interaction.

• Research Motivation and Related Work:

  • Many studies have explored the potential of using smartphones for on-plane or as-plane interaction in VR environments, but less attention has been given to "through-plane interaction," which involves interacting with distant targets through a transparent plane.
  • The issue of binocular parallax in transparent interfaces has persisted for a long time. Existing solutions, such as target position repetition, are unavoidable but still have limitations, necessitating new techniques to fundamentally address the problem.

Solution

• Proposed Method or Solution:

  • A new selection technique combining "Stereoscopic Viewing" and "Monoscopic Touching" is proposed.
  • Users can observe distant objects through the smartphone screen and precisely select these objects by touching the screen.
  • Smooth animation transitions between the two modes are introduced to reduce visual discomfort.

• Innovative Aspects:

  • Monoscopic rendering is applied to transparent mobile devices for the first time, and animation transitions are used to address interaction issues caused by binocular parallax.
  • A faster, more accurate, and intuitive distant selection technique is achieved, outperforming the widely used ray-casting technique.

• Implementation Steps and Key Techniques:

  1. The smartphone defaults to "Stereoscopic Viewing Mode," presenting itself as a transparent glass plane, allowing users to observe distant targets through the screen.
  2. When the user's finger approaches the screen, it switches to Monoscopic Touching Mode, displaying the screen content with monoscopic rendering to eliminate binocular parallax issues.
  3. Smooth animation transitions between the two modes are implemented to prevent sudden visual changes that could cause eye fatigue.
  4. An algorithm dynamically adjusts the 3D mesh spatial position of the target objects to achieve animation transitions.

Research Outcomes

• Specific Results:

  • The proposed technique significantly improved target selection time and accuracy: target selection time was reduced by 27%, and selection accuracy increased by 45%.

• Advantages Compared to Existing Solutions:

  • Compared to ray-casting, this technique performs better in distant target selection, particularly as the target distance increases.
  • Animation transitions reduce visual fatigue and enhance user interaction experience.

• Experimental or Evaluation Results:

  • Quantitative experiments with 12 participants and NASA TLX questionnaire surveys revealed:
    • The new technique significantly outperformed ray-casting in terms of target selection time and accuracy.
    • The new technique scored better in user-perceived mental workload, frustration, and performance.

• Limitations and Future Directions:

  • Limitations:
    • The prototype implementation uses a transparent acrylic board to simulate a smartphone, which may limit users' grip posture options.
    • The current method of detecting touch events through sound has issues with false triggers, though it minimally affects experimental results.
  • Future Directions:
    • Explore designs for more comfortable long-term use and repetitive tasks.
    • Model its interaction performance for quantitative analysis of application scenarios, such as using Fitts' Law for performance modeling.
    • Expand the technology to touch devices or walls of different sizes.
    • Support stylus input to enhance usage in complex scenarios like 3D sketching.

This paper provides a robust new perspective on interaction technology in the VR field, emphasizing intuitive transparent interaction through smartphone screens to enrich operational possibilities in VR spaces.