SafeSpect: Safety-First Augmented Reality Heads-up Display for Drone Inspections

Research Background and Problem

• Identified Problems or Challenges: The authors highlight significant issues with current tablet-based interfaces for drone operations, including high cognitive load for pilots and divided attention between video feeds and the real world, which reduces situational awareness. In tasks such as building facade inspections, these issues may lead to safety risks, such as drone collisions or operational errors. Furthermore, the authors emphasize that existing research primarily focuses on specific interface elements (e.g., camera views, flight paths) while paying less attention to safety-critical scenarios.
• Importance: As drones are increasingly deployed in dense urban environments, safety concerns become particularly critical. Pilots must maintain safety awareness during operations while processing complex information to identify building defects. This necessitates interfaces that effectively balance task-related and safety-related information. Enhancing situational awareness and reducing cognitive load are key to ensuring the safety of drone operations.
• Research Motivation and Related Work: Previous studies have demonstrated that AR can improve situational awareness. However, directly transferring 2D interfaces to head-mounted displays may increase visual burden. The authors aim to optimize AR design to address information overload and enhance operator safety awareness in safety-critical scenarios.

Solution

• Proposed Method or Solution: The authors propose a task-dynamically adaptive augmented reality (Adaptive AR) interface, designing a system with adaptive view switching that distributes information between task views and safety views. This approach aims to avoid cognitive overload while enhancing situational awareness and operational efficiency during building facade inspection tasks.
• Innovations: Compared to static AR or current 2D interfaces, the proposed adaptive AR interface offers significant innovations: dynamically adjusting information display based on task requirements, reducing visual distractions by hiding non-essential information, and iteratively optimizing the design through participatory input from professional pilots.
• Implementation Steps and Key Technologies:
  1. Design Phase: Conduct design workshops and iterative development in collaboration with professional drone pilots to identify key issues and design requirements.
  2. Interface Development: Build a virtual reality (VR) simulator using Unity and Meta Quest 3, integrating task-related and safety-related information.
  3. Key Technologies:
     • Task views include 3D visualizations of flight paths and building coverage.
     • Safety views display real-time information on drone positioning, heading, GPS errors, and return-to-home (RTH) paths.
     • Sensors and dynamic adjustment mechanisms accurately present safety alerts (e.g., low battery or GPS failure) and enhance user response to safety issues through automatic format switching.

Research Outcomes

• Specific Outcomes:

  • The adaptive AR interface significantly reduced pilots' cognitive load and improved situational awareness compared to current 2D and static AR interfaces.
  • The task view allowed users to focus more effectively on building inspection tasks, while the safety view provided real-time alert information during hazardous situations.
  • Experimental results indicate that the adaptive interface positively impacted pilots' efficiency in switching between information and their overall safety.

• Advantages Over Existing Solutions:

  • Compared to traditional 2D interfaces, the adaptive AR interface significantly reduced cognitive load.
  • Compared to static AR interfaces, it dynamically prioritized critical information, avoiding visual distractions.
  • In practical evaluations, the adaptive interface more effectively optimized the safety and efficiency of drone operations.

• Experiment or Evaluation Results:

  • Statistical data showed that the adaptive AR interface reduced cognitive load scores (from 5.40 for the 2D interface to 4.13 for the adaptive interface).
  • It improved situational awareness scores (SART increased from 17.47 to 24.07).
  • User confidence in the interface and accessibility of safety information significantly improved.
  • While objective task performance (e.g., percentage of correctly marked defects) showed no significant differences across interfaces, subjective user satisfaction with task completion and outcomes improved significantly.

• Limitations and Future Directions:

  • Limitations:
    • The experiment primarily involved amateur drone users. Although the virtual environment simulated real tasks, challenges such as lighting variations and AR device technical issues (e.g., tracking drift) in real-world environments remain unresolved.
    • The adaptive interface may cause initial unfamiliarity during the learning phase, and hidden information could lead to user distrust.
  • Future Research Directions:
    • Increase participation of professional drone pilots to explore the interface's impact on users with varying experience levels.
    • Test the interface in real-world drone operation scenarios while optimizing AR visibility to accommodate environmental lighting and the complexity of real-world conditions.
    • Enhance interface customization capabilities, allowing users to adjust information presentation to suit personal preferences. Develop user-controllable adaptive behavior schemes to further improve system transparency and trustworthiness.