Exploring the Design Space of Optical See-through AR Head-Mounted Displays to Support First Responders in the Field

Title of the Paper

Exploring the Design Space of Optical See-through AR Head-Mounted Displays to Support First Responders in the Field

Paper Information

• Subject Area: Applications of Augmented Reality Technology in Emergency Response Scenarios
• Keywords: Augmented Reality (AR), Design, Field Tasks, Head-Mounted Displays, First Responders

Research Background and Issues

• Identified Problems or Challenges:

  • First responders (e.g., firefighters, law enforcement officers, and medical service providers) often face high-risk tasks in emergency scenarios, including making rapid decisions, interacting with dynamic environments, and collaborating with team members.
  • Existing technological support is limited, such as relying solely on traditional mobile radios for communication, lacking advanced tools to provide real-time information and support.
  • While mobile augmented reality (AR) has seen some applications in emergency response, head-mounted AR technology has not yet been widely explored.

• Research Significance:

  • AR head-mounted displays (HMDs) can overlay virtual information onto real-world environments in a hands-free manner, potentially improving operational efficiency and safety for first responders.
  • Gaining a deeper understanding of first responders' needs through direct engagement is key to developing effective AR technologies.

• Research Motivation and Related Work:

  • Literature review indicates that different types of first responders (firefighters, law enforcement officers, medical service providers) face distinct task environments and requirements. Therefore, customized AR solutions are necessary.
  • Existing research on AR technology for emergency response primarily focuses on mobile devices, leaving the opportunities and challenges of head-mounted devices underexplored.

Solution

• Proposed Methods or Solutions:

  • Using Microsoft HoloLens 2 as the research platform to provide first responders with direct experiences, including hardware evaluation, multimodal interaction technologies, and four types of augmented reality prompts (overview prompts, directional prompts, highlighting prompts, and annotation prompts).
  • Conducting semi-structured interviews with 26 first responders, including firefighters, law enforcement officers, and medical service providers, to gain in-depth insights into the needs of different roles.

• Innovations:

  • Providing first-hand experiences for first responders to optimize design.
  • Integrating hardware, interaction technologies, and visual feedback to comprehensively outline the design space of AR technology in emergency scenarios.
  • Analyzing role- and task-specific AR requirements for first responders and proposing recommendations for multi-role collaboration.

• Implementation Steps and Key Technologies:

  1. Hardware Evaluation: Participants wear the HoloLens 2, simulate field task scenarios, and evaluate comfort, field of view, and compatibility with existing equipment.
  2. Exploration of Interaction Technologies: Participants experience multimodal input methods such as air gestures, voice commands, and gaze interactions, discussing their potential applications and limitations.
  3. Discussion of AR Visual Prompts: Participants discuss how to design and optimize maps, directional prompts, highlighting prompts, and annotation prompts to support field tasks.

Research Outcomes

• Specific Outcomes:

  • Role-specific requirements for different emergency task scenarios (e.g., firefighters needing path retracing markers, ambulance drivers requiring low-distraction navigation prompts).
  • Designed recommendations for multi-layered maps suitable for emergency scenarios: a base layer displaying structural information, with dynamic layers for on-demand information.
  • Proposed optimized designs for directional, highlighting, and annotation prompts based on actual needs (e.g., highlighting interactive features of objects rather than entire objects).

• Advantages Compared to Existing Solutions:

  • Compared to mobile augmented reality, head-mounted AR devices enable hands-free operation, helping first responders stay focused on primary tasks.
  • Considered role- and scenario-specific needs, such as enhancing the operability of objects (e.g., indicating door-opening directions) to support rapid actions.

• Experimental or Evaluation Results:

  • Discussed the applicable scenarios and design details of various augmented reality prompts, including color, thickness, transparency, and annotation positioning.
  • Emphasized the brevity and uniqueness of voice commands, as well as the need for technology to adapt to noisy environments.
  • Extracted design principles suitable for real-time task scenarios based on feedback from participants' functional evaluations.

• Limitations and Future Directions:

  1. The experiments were primarily conducted in idealized laboratory settings, which may not fully reflect the challenges of real-world field environments. Future research should include more field experiments.
  2. Gender imbalance among participants (predominantly male) may limit the study's coverage of the needs of female first responders.
  3. Participant roles were heavily weighted toward firefighters; future studies should further expand to include personnel from other emergency response fields.

The above provides a structured summary and key points, offering clear analysis and insights. Further discussion can be conducted if needed.