Exploring the Impacts of Background Noise on Auditory Stimuli of Audio-Visual eHMIs for Hearing, Deaf, and Hard-of-Hearing People

Paper Title

Exploring the Impacts of Background Noise on Auditory Stimuli of Audio-Visual eHMIs for Hearing, Deaf, and Hard-of-Hearing People

Publication Info

• Topic area: Accessibility and usability of external human-machine interfaces (eHMIs) in automated vehicles for diverse pedestrian groups.
• Keywords: eHMI, automated vehicles, auditory stimuli, background noise, Deaf and Hard-of-Hearing (DHH), virtual reality, accessibility, trust, perceived safety, multi-modal interfaces.

Background and Problem

• Problem / challenge: Existing eHMI research rarely includes disabled pedestrians, particularly DHH individuals, and overlooks how auditory stimuli perform under varying background noise levels. This creates accessibility barriers and potential inequities in transport.
• Significance: Ensuring eHMI designs are inclusive and effective for all pedestrians, including DHH individuals, is crucial for equitable and safe automated vehicle-pedestrian interactions.
• Motivation and related work: Prior studies have explored multi-modal eHMIs but have not adequately addressed the needs of DHH pedestrians or the impact of urban background noise on auditory stimuli. This study aims to fill these gaps by evaluating audio-visual eHMIs in realistic soundscapes.

Solution

• Proposed approach: A virtual reality (VR) simulation study to assess the effects of background noise (quiet vs. loud) and auditory stimuli (baseline, bell, speech) on crossing experiences and behaviors of Hearing and DHH pedestrians.
• Novelty:
  1. First empirical VR evaluation of auditory stimuli under different noise levels for Hearing and DHH pedestrians.
  2. Inclusion of DHH participants to identify accessibility barriers in auditory eHMI design.
  3. Examination of multi-modal eHMI effectiveness in urban soundscapes.
  4. Practical implications for inclusive eHMI design.
• Procedure and key techniques:
  • Developed a VR environment simulating a two-lane urban road with automated vehicles (AVs).
  • Tested two background noise levels (quiet: ~58 dB; loud: ~70 dB) and three auditory stimuli (baseline, bell, speech).
  • Measured subjective (trust, acceptance, perceived safety, mental workload) and objective (eye gaze, step-in time, early step-in count) outcomes.
  • Recruited 25 Hearing and 11 DHH participants, all using hearing technologies, for a within-subjects study.

Results

• Concrete findings:
  • Loud background noise significantly impaired subjective crossing experiences (trust, usefulness, safety, mental workload) but did not affect crossing behaviors.
  • Auditory stimuli (bell and speech) improved subjective experiences but had no measurable impact on crossing behaviors.
  • DHH participants showed greater variability in auditory eHMI perception, with only 2 of 11 fully perceiving both auditory stimuli under all conditions.
• Advantage over baselines:
  • Bell and speech eHMIs were rated higher in trust, usefulness, and safety compared to the baseline (vehicle sounds only).
  • Speech was ranked slightly higher than bell overall but showed mixed results for DHH participants.
• Experiments / evaluation:
  • Conducted 432 trials (36 participants × 6 conditions × 2 repetitions).
  • Used non-parametric statistical analyses to account for small and unequal group sizes.
  • Included qualitative interviews to capture participant perceptions and preferences.
• Limitations and future work:
  • Small sample size for DHH participants (N=11) and limited ecological validity due to controlled VR scenarios.
  • Did not explore sudden or momentary sounds or other disabled groups (e.g., visually impaired).
  • Future work should include longitudinal studies, diverse cultural contexts, and collaboration with hearing technology companies.

Summary

This study evaluated the impact of background noise and auditory stimuli on the crossing experiences and behaviors of Hearing and DHH pedestrians using a VR simulation. Results showed that loud background noise negatively affected subjective experiences but not behaviors, and auditory stimuli (bell and speech) improved experiences without influencing behaviors. DHH participants exhibited variability in auditory eHMI perception, highlighting the need for inclusive designs. The findings underscore the importance of multi-modal eHMIs and call for further research to address accessibility barriers, particularly for DHH individuals. Four practical implications were proposed to guide future eHMI design and research.