Perspective and Geometry Approaches to Mouse Cursor Control in Spatial Augmented Reality

Document Title

Perspective and Geometry Approaches to Mouse Cursor Control in Spatial Augmented Reality

Document Information

• Subject Area: Interaction techniques in Spatial Augmented Reality (SAR); Human-Computer Interaction
• Keywords: Interaction techniques, Spatial Augmented Reality, mouse control, geometric computation, perspective cursor, user study

Research Background and Issues

• Identified Problems or Challenges:

  • SAR environments use projection technology to map digital information onto real-world object surfaces, offering vast application potential. However, extending traditional mouse input methods to complex 3D surfaces (e.g., irregular surfaces and curved surfaces) poses challenges, including geometric irregularities, gaps between surfaces, and occlusion issues.
  • Existing perspective-based mouse cursor control methods (e.g., perspective cursor and standard ray projection) struggle with achieving precise and continuous cursor movement. While efficient for long-distance movements, these methods exhibit high error rates in small-scale tasks.

• Research Importance:

  • The mouse, as a familiar input tool in desktop user interfaces, can enhance interaction precision and integration in SAR environments, providing convenience and consistency for the widespread adoption of SAR technology.

• Research Motivation:

  • To explore innovative solutions that balance cursor movement efficiency (speed) and precision, while improving user experience in practical scenarios involving complex geometric surfaces.

Solution

• Proposed Methods and Solutions:

  • Everywhere Cursor: A geometric-based cursor control technique is proposed, enabling smooth cursor movement along complex 3D surfaces through relative mouse movement. This technique combines directed projection around the cursor with surface normal information extraction.
  • Comparative experiments were conducted between this geometric method and two existing perspective methods (perspective cursor and standard ray projection) to evaluate accuracy, speed, and user subjective experience.

• Innovations:

  • Defined and implemented the first geometric mouse cursor control technique tailored for SAR.
  • Introduced a six-direction normal adjustment mechanism for the cursor, using local averaging to estimate the cursor's "facing direction" on the surface.
  • Developed interactive "gap-bridging tools" and "projected cursor models" to enable the cursor to traverse gaps and occlusions on irregular surfaces.
  • Supported seamless transitions of the mouse cursor between monitor displays and SAR environments, enhancing the continuity of user interaction.

• Implementation Steps/Key Techniques:

  1. Cursor Normal Projection and Direction Correction: Six short rays are projected around the cursor to calculate the average direction of local surface normals, ensuring the cursor aligns with the current interactive surface.
  2. Circular Ray Projection Method: The mouse movement path generates circular ray projections along the movement direction to prevent the cursor from detaching from the surface.
  3. Smooth Transitions Across Surfaces: Interactive gap-bridging tools use virtual meshes to fill surface gaps, ensuring continuous cursor paths.
  4. Integration with Perspective Cursor: Ray projection is introduced as a backup mechanism to resolve issues of cursor sticking or loss.

Research Outcomes

• Specific Results:

  • Short-Distance Selection Tasks: Compared to perspective methods, "Everywhere Cursor" significantly reduced errors (by 17%-60%) and minimized path deviation on irregular and curved geometries.
  • Long-Distance Cursor Movement: Perspective cursors demonstrated faster speeds in long-distance tasks (22%-49% faster) but had higher error rates.
  • Trajectory Tasks (Precision Testing): The geometric method achieved 29%-60% higher accuracy than perspective methods, particularly on complex surfaces.
  • User Satisfaction and Subjective Evaluation: Users found the geometric method more natural and closer to the traditional desktop computing experience, performing more smoothly in detailed tasks and on irregular surfaces.

• Advantages Over Existing Solutions:

  • Improved cursor control accuracy on irregular surfaces and reduced mouse trajectory jitter.
  • Provided linear control similar to traditional cursors, enhancing intuitiveness.
  • The complementary design of the two methods made the system more adaptable to mixed work scenarios.

• Experimental Limitations and Future Directions:

  • Limitations:
    • Experiments primarily used a controlled environment with a single fixed set of objects, limiting broader applicability to other diverse environments.
    • The weight of the mouse may have impacted the performance of perspective methods.
    • Participants were predominantly young, right-handed mouse users, leaving universal applicability across all demographics unexplored.
  • Future Directions:
    • Investigate machine learning algorithms for automatically filling surface gaps in larger-scale scenarios.
    • Study "speed-adjustable" hybrid techniques that combine the speed advantage of perspective methods with the precision of geometric methods.
    • Test dynamic mechanisms for cursor correction in open-ended tasks to further optimize cursor direction adjustment rules.

Conclusion

This study expands the possibilities of mouse cursor control in SAR environments by proposing a novel geometric-based method and validating its performance advantages on complex geometric surfaces through experiments. Future work will continue to explore the integration of perspective and geometric methods to further optimize user interaction experiences and promote the widespread application of SAR technology.