HoloBar: Rapid Command Execution for Head-Worn AR Exploiting Around the Field-of-View Interaction

Document Title

HoloBar: Rapid Command Execution for Head-Worn AR Exploiting Around the Field-of-View Interaction

Document Information

• Subject Area: Augmented Reality (AR), Human-Computer Interaction Design
• Keywords: Augmented Reality, Smartphone Interaction, Menu Interaction, HoloLens, Microinteraction

Research Background and Problem

• Identified Problems or Challenges:

  • Current menu interfaces on head-mounted displays (HMDs) are inefficient, leading to cumbersome command execution.
  • Menus displayed within the limited field of view of HMDs can obstruct content and restrict menu size.
  • Air gesture-based menu interactions are difficult to operate precisely and can cause arm fatigue.

• Significance:

  • Enhancing command execution efficiency on HMDs is crucial for improving user experience in augmented reality applications.
  • Improving menu interaction can make operations smoother and reduce task interruptions.

• Research Motivation and Related Work:

  • Combining smartphones with HMDs to host interaction processes can leverage the touchscreen and precise input capabilities of smartphones to compensate for HMD limitations.
  • Existing research focuses primarily on single tasks such as navigation or 3D exploration, with limited studies on complete command execution (including menu activation, item selection, and validation).

Solution

• Proposed Method or Solution:

  • HoloBar is a novel two-level menu interaction tool that distributes the menu system across the bottom of the HMD field of view and the smartphone screen.
  • Menu activation and item selection are performed using smartphone interaction around the HMD, with command validation via the touchscreen.

• Innovations:

  • Rapid menu activation through interaction around the field of view without pausing the current HMD application.
  • Implicit top-level menu item selection achieved by horizontal smartphone movement, reducing selection time.
  • Secondary menu item preview on the smartphone screen, minimizing HMD content obstruction and enhancing visual search efficiency.

• Implementation Steps and Key Technologies:

  1. Menu Layout: The top-level menu is fixed as a menu bar at the bottom of the HMD field of view, while the secondary menu is displayed on the smartphone screen.
  2. Menu Activation: Users activate the menu by bringing the smartphone to the bottom of the HMD field of view.
  3. Item Selection and Validation: Users move the smartphone to select top-level menu items and use the touchscreen to select and validate secondary menu items.
  4. Technical Implementation: An accurate infrared optical tracking system detects the smartphone's pose relative to the HMD.

Research Outcomes

• Specific Results:

  • HoloBar demonstrated higher efficiency across all steps of command execution (menu activation, top-level menu item selection, secondary menu item selection, and command validation).
  • User experience showed a success rate of over 99%, effectively supporting up to 80 command items.
  • The average time for users to execute a complete command was 2.4 seconds, meeting microinteraction requirements.

• Advantages:

  • Compared to existing HoloLens menus, HoloBar offers a 29% time-saving advantage.
  • Superior performance in user experience and command execution fluency.

• Experimental or Evaluation Results:

  • Experiments indicated that a menu activation distance of 20° provides the optimal balance between performance and fatigue.
  • Comparative analysis of different menu designs showed that HoloBar significantly outperforms head-based or hand-based menu interactions in terms of efficiency and accuracy.
  • HoloBar's touch-based validation method is more efficient than HoloLens' air gesture or clicker-based validation methods.

• Limitations and Future Directions:

  • Lower user preference is associated with smartphone weight, which can be addressed by adopting lighter devices.
  • Designing multiple HoloBar switching mechanisms for system-level and in-app commands.
  • Future work could expand menu bar layouts using fisheye techniques or grid formats to accommodate more menu items.
  • Long-term adaptability and effectiveness of HoloBar can be validated in industrial augmented maintenance tasks.