Towards Inclusive External Human-Machine Interface: Exploring the Effects of Visual and Auditory eHMI for Deaf and Hard-of-Hearing People

Paper Title

Towards Inclusive External Human-Machine Interface: Exploring the Effects of Visual and Auditory eHMI for Deaf and Hard-of-Hearing People

Publication Info

• Topic area: Accessibility in automated vehicle-pedestrian interaction
• Keywords: External human-machine interface, automated vehicles, Deaf and Hard-of-Hearing, visual eHMI, auditory eHMI, accessibility, virtual reality, pedestrian safety, inclusive design, multi-modal communication

Background and Problem

• Problem / challenge: Existing eHMI designs for automated vehicles (AVs) largely exclude considerations for Deaf and Hard-of-Hearing (DHH) people, despite their unique needs and the increased risks they face in road crossing scenarios.
• Significance: Addressing this gap is critical for ensuring equitable access to AV-pedestrian communication, reducing mobility inequities, and enhancing road safety for DHH individuals.
• Motivation and related work: Prior studies have explored eHMI accessibility for other disabled groups (e.g., visually impaired, wheelchair users) and suggested multi-modal designs. However, research on DHH people’s interaction with eHMIs is sparse, particularly regarding their perception of visual and auditory cues. This study builds on existing literature to address this gap.

Solution

• Proposed approach: A two-phase study involving focus groups and a virtual reality (VR) user study to evaluate visual and auditory eHMI designs for DHH people.
• Novelty:
  1. First focus group study with DHH individuals and stakeholders to identify design requirements and select visual eHMI candidates.
  2. First VR study evaluating the impact of visual (Abstract Light, Abstract Light + Text, Abstract Light + Symbol) and auditory (With Speech, Without Speech) eHMIs on DHH and hearing participants.
  3. Identification of five practical implications for inclusive eHMI design.
• Procedure and key techniques:
  • Focus group study: Two groups (DHH and stakeholders) evaluated six visual eHMI concepts and provided qualitative and quantitative feedback on trust, acceptance, perceived safety, and mental workload.
  • VR user study: 32 participants (16 DHH, 16 hearing) interacted with AVs in a simulated urban environment, assessing crossing experience (trust, acceptance, perceived safety, workload) and crossing behavior (gaze duration, step-in road time, early step count).
  • Analysis: Mixed ANOVA and thematic analysis to compare group-specific responses and identify design preferences.

Results

• Concrete findings:
  • Visual eHMIs (Abstract Light + Text, Abstract Light + Symbol) significantly improved trust, usefulness, perceived safety, and crossing behavior compared to No Visual.
  • Speech-based auditory eHMI enhanced trust and perceived safety but did not significantly impact crossing behavior.
  • DHH participants spent more time looking at AVs than hearing participants, indicating heightened caution.
• Advantage over baselines:
  • Combined-visual eHMIs (Abstract Light + Text, Abstract Light + Symbol) outperformed singular visual designs (Abstract Light) and No Visual in subjective ratings and gaze behavior.
  • Auditory eHMI improved crossing experience compared to Without Speech but showed variability in effectiveness for DHH participants.
• Experiments / evaluation:
  • Datasets: 512 trials across eight conditions (4 visual × 2 auditory).
  • Metrics: Trust, acceptance, perceived safety, workload (pupillary activity), gaze behavior, step-in road time, early step count.
  • Participants: 32 individuals (16 DHH, 16 hearing), diverse in age, gender, and hearing levels.
• Limitations and future work:
  • Controlled VR environment limits ecological validity; future studies should explore complex road scenarios and mixed-traffic settings.
  • Focused on Abstract Light as the singular visual design; additional visual and auditory concepts (e.g., non-verbal sounds, haptics) should be tested.
  • Study conducted in the UK; cross-cultural validation is needed.

Summary

This study addresses the lack of inclusive eHMI designs for DHH pedestrians by evaluating visual and auditory communication methods in AV-pedestrian interactions. Focus groups identified key design requirements, and a VR user study demonstrated that combined-visual eHMIs (Abstract Light + Text, Abstract Light + Symbol) improve crossing experience and behavior for both DHH and hearing participants. Speech-based auditory eHMIs enhanced trust and perceived safety but showed limited impact on crossing behavior, especially for sign language users. The findings highlight the need for multi-modal and combined-visual eHMIs to ensure accessibility and inclusivity. Future work should explore more diverse scenarios, modalities, and participant groups.