When Audio Is Enough: Design-Tradeoffs in Multi-Story MR Navigation

Paper Title

When Audio Is Enough: Design-Tradeoffs in Multi-Story MR Navigation

Publication Info

• Topic area: Mixed Reality (MR) navigation using audio-centric interfaces.
• Keywords: Mixed Reality, Audio AR, Spatial Cognition, Cognitive Load, Navigation Efficiency, Environmental Awareness, Multi-Story Navigation, Spatial Audio, Visual AR, Augmented Reality.

Background and Problem

• Problem / challenge: MR interfaces predominantly rely on visual modalities, which can lead to attentional tunneling, reduced environmental awareness, and high cognitive load, especially in complex 3D environments like multi-story buildings. Prior research has focused on 2D navigation or single-floor 3D navigation, leaving vertical navigation and multi-story environments underexplored.
• Significance: Addressing these challenges could improve MR usability in safety-critical applications, accessibility for visually impaired individuals, and training for first responders.
• Motivation and related work: Previous studies have shown that spatial audio can enhance realism and user experience in stationary or 2D navigation contexts. However, the cognitive challenges of multi-story navigation, including verticality and occlusion, remain largely unaddressed. This study builds on prior work by investigating audio-centric MR navigation in complex indoor environments.

Solution

• Proposed approach: An Audio-First Mixed Reality navigation framework leveraging spatialized audio cues, combined with Head-Related Transfer Functions (HRTF), to guide users in multi-story environments.
• Novelty:
  1. Extends Audio AR (AAR) research into multi-story indoor navigation.
  2. Compares four conditions: no AR aid, VisualOnlyAR, AudioOnlyAR, and AudioVisualAR.
  3. Investigates cognitive trade-offs between audio and visual guidance in MR navigation.
  4. Provides design implications for audio-centric MR systems.
• Procedure and key techniques:
  • Participants navigated a three-floor building under one of four conditions.
  • Spatialized audio cues were generated using Steam Audio and a digital twin of the building.
  • Environmental awareness was assessed through object recognition tasks.
  • Cognitive load was measured using NASA TLX questionnaires.
  • Quantitative metrics included navigation time, distance traveled, and object recognition accuracy.

Results

• Concrete findings:
  • AudioOnlyAR and VisualOnlyAR improved navigation efficiency compared to no-AR, with VisualOnlyAR performing slightly better overall.
  • AudioOnlyAR users reported the lowest mental demand, while AudioVisualAR users experienced the highest cognitive load.
  • Environmental awareness was highest for AudioOnlyAR and no-AR users, while VisualOnlyAR and AudioVisualAR reduced awareness due to attentional tunneling.
• Advantage over baselines:
  • AudioOnlyAR was non-inferior to VisualOnlyAR in navigation efficiency and environmental awareness.
  • AudioOnlyAR reduced mental demand compared to hybrid AudioVisualAR systems.
• Experiments / evaluation:
  • 28 participants (7 per group) completed four navigation tasks in a real building.
  • Metrics included task completion time, distance traveled, and object recognition accuracy.
  • Statistical analyses included ANOVA and non-inferiority testing.
• Limitations and future work:
  • Small sample size limited statistical power.
  • Tasks may have varied in difficulty, affecting comparability.
  • Future work should explore personalized HRTFs, larger sample sizes, and applications in high-stakes scenarios like first response or accessibility for visually impaired users.

Summary

This study explored the role of audio-centric MR navigation in multi-story environments, comparing AudioOnlyAR, VisualOnlyAR, AudioVisualAR, and no-AR conditions. Results showed that AudioOnlyAR is non-inferior to VisualOnlyAR in navigation efficiency, reduces cognitive load, and maintains environmental awareness. VisualOnlyAR caused attentional tunneling, while AudioVisualAR overwhelmed users. Findings suggest that audio interfaces can enhance MR usability in complex indoor spaces, with implications for accessibility and safety-critical applications. Future research should address sample size limitations, task difficulty calibration, and real-world deployment scenarios.