Next-Generation Navigation: Evaluating the Impact of Augmented Reality on Situation Awareness in General Aviation Cockpits

Research Background and Problem

• Identified Problem:
  Traditional General Aviation (GA) flights typically rely on 2D map navigation, which forces pilots to frequently shift their attention between inside and outside the cockpit, thereby reducing their situational awareness (SA). This switching not only diminishes real-time monitoring of the external environment but may also compromise safety.

• Significance of the Research:

  • The accident rate per flight hour in GA is significantly higher than in commercial aviation, partly because technological innovations in commercial aviation focus on large aircraft while neglecting the needs of GA pilots.
  • Delayed innovation has resulted in GA pilots lagging behind consumer technology research, with AR applications in GA still in their infancy.
  • Enhancing situational awareness and reducing repetitive attention shifts are crucial for GA flight safety.

• Research Motivation:

  • Existing literature indicates that Augmented Reality (AR) effectively improves safety and performance in military and commercial aviation, but research in the GA domain remains sparse.
  • Preliminary work has demonstrated the feasibility of AR technology, including reducing yaw errors by displaying flight information.
  • Expanding existing AR solutions to the GA domain to enhance pilots' situational awareness.

Solution

• Proposed Solution:
  Develop and test an AR tool based on Microsoft HoloLens 2 that displays relevant information such as landmarks, airspace structures, flight paths, and aviation obstacles. The tool is tested in a simulator to evaluate its impact on pilots' situational awareness, workload, and flight accuracy.

• Innovations:

  • Directly projecting invisible or hard-to-detect navigation and airspace information into the pilot's field of view, reducing frequent attention shifts between inside and outside the cockpit.
  • For the first time, extending situational awareness research for GA pilots to dynamic flight scenarios (e.g., landing paths).

• Implementation Steps:

  1. System Architecture:
     • Develop an AR application using Unity to meet real-time visual requirements.
     • The application displays information via the Microsoft HoloLens 2 head-mounted display, integrating data on aircraft position, airspace, and obstacles from flight simulation software (Microsoft Flight Simulator 2020).
  2. Data Visualization:
     • Visualize four types of information: Points of Interest (POI), airspace structures, flight routes, and aviation obstacles (e.g., cables, towers).
     • Provide semantic scene displays with automatic adjustments for near and far perspectives.
  3. Interaction Design:
     • Adjustable display range (zoom in/out) and toggle specific information on/off.
     • Functional menu positioned at the cockpit side to avoid interfering with flight operations.
  4. Experimental Methodology:
     • Deploy the tool in a flight simulator with real-world scenarios, testing its performance with 19 certified pilots across multiple experimental tasks.

Research Outcomes

• Specific Results:

  • Position Recognition (Task 1):

    • In areas with inconspicuous terrain features, AR significantly reduced pilots' error distance in estimating their location.
    • AR quickly presented subtle landmark features, significantly improving location accuracy.
    • In terrain-rich areas, AR assistance was effective but less significant.

  • Airspace Understanding (Task 2):

    • AR increased the correct answer rate from 68.4% without AR to 97.4%, while significantly reducing response time.
    • Pilots were able to understand current and future airspace structures more quickly and accurately.

  • Trajectory Accuracy (Task 3):

    • No significant improvement in flight path Frechet distance (deviation), possibly influenced by terrain differences and visibility.
    • A trend toward more standardized flight trajectories was observed in certain scenarios (e.g., simple terrain areas).

  • Situational Awareness and Workload:

    • Situational awareness showed no significant improvement, particularly in areas with distinct terrain features, and data "redundancy" might have caused slight distraction.
    • Overall workload (NASA-TLX) did not change significantly, though physical burden slightly decreased, while mental demand and interference slightly increased.

  • User Feedback:

    • Advantages:
      • Improved location recognition, helping pilots quickly identify the environment in unfamiliar airports or low-visibility conditions.
      • Intuitive display of airspace structures reduced the likelihood of pilots making "prediction errors."
    • Challenges and Areas for Improvement:
      • Information occasionally "blocked" the actual external view (reported by 4 pilots).
      • Users suggested dynamically switching display content based on flight phases to avoid visual interference.
      • Over-reliance on AR might lead to degradation of traditional operational skills.

• Comparison with Existing Solutions:
  Compared to standalone navigation designs or traditional 2D charts, the proposed AR tool demonstrated smoother handling of complex structures but currently lacks adaptability to other flight conditions (e.g., low light, emergency states).

• Experimental Limitations and Future Directions:

  • Limitations:
    1. Current tests are simulator-based and do not include sensory feedback from real flights (e.g., vibration, sound).
    2. The field of view of head-mounted devices is limited, potentially affecting comprehensive scene visibility.
    3. Sample size (19 pilots) restricts the generalizability of results.
  • Future Research Directions:
    1. Enhance hardware performance and conduct tests in actual flight scenarios.
    2. Optimize interaction experience through visual design improvements for AR icons and airspace color coding.
    3. Incorporate more complex scenarios (e.g., weather interference, emergency states) to validate algorithm applicability.
    4. Expand experiments to pilots with varying experience levels to assess learning curves and adaptability.

This series of experiments preliminarily confirms the strong potential of AR, but achieving widespread practical application requires further development in hardware compatibility, user interface optimization, and more realistic testing environments.