Eye-Perspective View Management for Optical See-Through Head-Mounted Displays

Title of the Paper

Eye-Perspective View Management for Optical See-Through Head-Mounted Displays

Paper Information

• Research Domain: Augmented Reality (AR), Optical See-Through Head-Mounted Displays (OST-HMD)
• Keywords: Augmented Reality, Optical See-Through, Head-Mounted Displays, Label Layout, Readability, Stereoscopic Vision, Synchronized View Management, Image Reprojection

Research Background and Problem Statement

• Identified Problems

  1. Optical see-through (OST-HMD) devices overlay augmented reality (AR) information onto real-world scenes via semi-transparent displays. However, this approach results in low contrast and poor readability due to the blending of the background with augmented information.
  2. Existing view management algorithms typically optimize label layout based on scene images captured by the device's built-in camera. However, the camera's perspective differs from the user's viewpoint, leading to layout errors in the user's actual view and negatively impacting readability.
  3. Most view management methods fail to adequately consider users' stereoscopic vision, where the background seen by each eye differs, further reducing label readability.

• Importance of the Research

  Addressing readability and contrast issues is critical for the practical application and user experience optimization of modern augmented reality devices. Solving the problem of accurately restoring the user's view in OST-HMDs can enhance user experience across various human-computer interaction scenarios.

• Research Motivation

  Proposes Eye-Perspective View Management based on the user's eye perspective to overcome the issue of camera viewpoint deviation. By leveraging high-fidelity reconstruction of background information in the user's actual view, the accuracy of layout computation and visual effects can be improved.

Solution

• Methodology and Core Ideas

  1. Eye-Perspective Rendering (EPR): Utilizes a real-time scene appearance reconstruction technique to synthesize high-fidelity rendered images from the user's binocular perspective.
  2. Optimized Label Placement: Adjusts label layout based on independent views from both eyes, incorporating multiple optimization criteria (background uniformity, brightness contrast, texture contrast, stereoscopic consistency, etc.) to enhance readability.

• Technical Innovations

  1. Introduced the first OST-HMD view management method centered on the user's actual view, providing a new paradigm that eliminates the limitations of camera viewpoint deviation in current algorithms.
  2. Proposed three implementation algorithms (homography-based, 3D reprojection-based, and image generation-based rendering methods) and selected the most suitable approach (reprojection method) based on practical performance requirements and hardware constraints.

• Implementation Steps and Technical Details

  1. Capture depth information and scene color using an RGBD camera.
  2. Convert the scene from the camera coordinate system to the user's binocular coordinate system, generating independent views for both eyes.
  3. Compute the optimal label placement based on predefined optimization criteria (e.g., background uniformity/brightness contrast).
  4. Ensure stereoscopic view consistency by evaluating differences between the two eyes' views to avoid visual discomfort caused by label layout.

Research Outcomes

• Specific Results

  1. User Experiments: Compared to traditional view management methods relying on built-in cameras, the proposed Eye-Perspective Rendering method significantly improved label placement accuracy, contrast, and readability.
  2. The EPR method demonstrated robustness across various complex backgrounds (including flat 2D and intricate 3D scenes) and is suitable for real-time operation on mobile devices.

• Advantages Over Existing Solutions

  1. Eliminates interference from camera viewpoint deviation in label layout, aligning more closely with the user's actual view.
  2. Enhances binocular view uniformity, reducing stereoscopic vision-induced "ghosting" effects.
  3. Achieves implementation through software improvements without requiring complex hardware modifications.

• Experimental and Evaluation Results

  • Label uniformity improved from 78% in traditional methods to 99.7%.
  • Subjective readability scores showed significant improvement.
  • In 97% of experimental cases, users preferred the label layout provided by the EPR method.

• Limitations and Future Directions

  1. Limitations
     • In certain scenarios, geometric occlusion between the user's eyes and the camera view may result in the loss of critical background information.
     • Experiments did not simulate scenarios involving extensive dynamic movement by users, leaving the method's adaptability to dynamic real-time scenes to be further validated.
  2. Future Directions
     • Develop mobile AR applications for use while walking.
     • Incorporate more complex background variations, such as dynamic and texture-overlapping scenes.
     • Extend algorithms to support additional task requirements, such as color adjustment and visual impairment assistance.

Feel free to request further analysis of experimental details or technical implementations!