Effects of Acoustic Transparency of Wearable Audio Devices on Audio AR

Research Background and Issues

• What problems or challenges did the authors identify?

  • In Audio Augmented Reality (AAR), devices must exhibit high acoustic transparency to ensure that users can naturally perceive real-world sounds while simultaneously adding virtual sounds. However, the relationship between acoustic transparency and the overall AAR experience, as well as its connection to users' subjective perceptions, remains unclear. Additionally, there is a lack of in-depth research on how different device designs affect the AAR experience.

• Why is this issue important?

  • With the widespread adoption of AAR technology, optimizing user experience is crucial. Devices need to overlay virtual sounds without obstructing real-world sounds while maintaining a comfortable experience. However, device design and transparency performance may influence user perception, posing new challenges for the development and application of AAR devices.

• Research motivation and related work:

  • Previous studies have evaluated the acoustic transparency of devices, focusing primarily on physical metrics or sound localization accuracy, without clarifying their relationship to users' subjective perceptions. The motivation of this research is to explore how laboratory-measured transparency impacts users' actual experiences, thereby optimizing AAR device design to enhance the quality of the experience.

Solution

• What methods or solutions did the authors propose?

  • The authors proposed a comprehensive research framework that analyzes the physical and subjective transparency of five devices with different shapes and transparency modes.
  • Subjective experiments include evaluating the spatial dimensions of real-world sound experiences (e.g., impact, source width, envelopment) and the overall impact of devices on the AAR experience.

• What is innovative about this solution?

  • The authors were the first to link physical transparency metrics (e.g., changes in Head-Related Transfer Function, HRTF) with users' subjective impressions. Additionally, they validated how high-transparency devices can provide enhanced live audio experiences through field demonstrations.
  • They used two types of AAR content in real-world environments (creating identical and distinct audio object streams) to provide practical use cases.

• What are the implementation steps? What key technologies were used?

  • Physical transparency measurement: Compared the power spectrum changes in the HRTF of different devices in worn and unworn states, calculating transparency indices through frequency adjustments.
  • Subjective experiments: Designed controlled environments for real and device-generated sounds to evaluate subjective transparency and perception metrics related to the AAR experience, such as naturalness, impact, and externalization.
  • Field demonstration experiments: Applied open-ear headphones in museums and multi-channel sound systems to validate the devices' enhancement effects and user satisfaction in real audio scenarios.

Research Outcomes

• What specific outcomes were achieved?

  • Relationship between device design and user experience:
    • Open-ear devices (e.g., Device E) performed best in terms of acoustic transparency and user perception, seamlessly integrating real and virtual sounds.
    • Devices with lower transparency (e.g., Devices A and D) tended to increase subjective unnaturalness and reduce attention to real-world sounds.
  • Perception of real-world sounds and AAR content:
    • Subjective experiments showed that the physical transparency of devices is highly correlated with users' perception of spatial dimensions (impact, width, envelopment).
    • With high-transparency devices, users experienced more natural interactions and were better able to focus on real-world sounds.

• What advantages does this solution have compared to existing ones?

  • The study provides a cross-dimensional analysis, including physical transparency, subjective transparency, and real-world scenario demonstrations. This comprehensive approach reveals the complex relationship between device performance and user experience.
  • It showcases new application cases, such as using open-ear devices to create more engaging augmented reality audio experiences in settings like museums and concerts.

• What were the experimental or evaluation results?

  • Subjective experiments demonstrated that high-transparency devices (e.g., open-ear headphones) reduced discomfort caused by device-generated sounds and improved overall user satisfaction.
  • Museum experiences showed that open-ear headphones could seamlessly integrate device-generated and live sounds, with a 93% satisfaction rate reflecting the technology's potential.
  • In the second experiment, combining commentary audio with music content optimized comprehension and experience quality, resulting in high overall satisfaction.

• Limitations and future directions

  • Limitations:

    • The study was limited by the number of devices tested (only five models), necessitating further expansion to include more device types.
    • The statistical analysis was constrained by the limited number of participants (only 15 in subjective experiments).
    • The study did not explicitly explore the potential synergistic effects of other sensory stimuli (e.g., visual, environmental sounds) on the AAR experience.

  • Future directions:

    • Develop more precise auditory models to predict users' subjective impressions of real and augmented sounds.
    • Explore methods for overlaying virtual sounds without interfering with real-world sounds.
    • Investigate the interaction effects of multi-user devices to optimize AAR experiences in public settings.