Light My Way. Developing and Exploring a Multimodal Interface to Assist People With Visual Impairments to Exit Highly Automated Vehicles

Research Background and Issues

Issues and Challenges

• The authors identified significant challenges in safety, environmental perception, and situational awareness faced by blind and visually impaired persons (BVIPs) after autonomous vehicles (HAVs) arrive at their destination.
• Unlike traditional vehicles, autonomous vehicles cannot provide driver guidance and support, leaving these users to navigate unfamiliar environments independently and face potential risks.
• Current accessibility technologies typically focus on other aspects of the journey (e.g., boarding and locating the vehicle), with limited research on how to safely exit autonomous vehicles.

Importance of the Problem

• Over 270 million people worldwide are affected by visual impairments, and population aging may further increase this number.
• Autonomous vehicle technology offers BVIPs the potential for independent travel, but ensuring they can safely adapt to new environments is crucial to avoid risks such as collisions and falls.

Research Motivation and Related Work

• Existing research primarily focuses on delivering traffic information during the journey (e.g., through tactile or audio feedback), neglecting the need for environmental awareness when exiting the vehicle.
• The ATLAS system proposed by Brinkley et al. only provides audio feedback to convey surrounding information. The authors believe that combining multimodal feedback (visual, auditory, and tactile) could be more effective.
• The authors aim to explore multimodal interfaces that can provide comprehensive environmental perception and adapt to the diverse needs of visually impaired users.

Solution

Proposed Method and Solution

• Developed PathFinder, a multimodal interface integrating visual, tactile, and auditory feedback to assist visually impaired individuals in understanding their surroundings safely after an autonomous vehicle has parked.
• Following participatory design principles, the authors involved users in the design process to ensure PathFinder meets the needs of users with varying degrees of visual impairment.

Innovations

• Combines tactile and audio feedback, with tactile providing a basic environmental overview and audio supplementing critical information.
• The interface design is simple and intuitive, featuring components such as a compass needle, expandable obstacle buttons, and scene audio notifications to enhance user experience.
• Offers both active information retrieval and passive information reception modes to accommodate different user habits.
• PathFinder's hardware design is reusable, with all construction files made open-source.

Implementation Steps and Key Technologies

1. Needs Assessment: Conducted interactive workshops with five BVIPs to identify their key information needs (e.g., dynamic obstacles, ground conditions, final destination).
2. Prototype Development: Designed three low-fidelity prototypes (smartphone, touchscreen window prototype, tactile bar prototype) to test the effectiveness of different information delivery methods and strategies.
3. System Construction: Developed PathFinder based on workshop results, featuring:
   • A compass needle indicating the direction of the final destination.
   • Expandable buttons providing tactile feedback and supporting audio feedback upon pressing.
   • Initial voice announcements summarizing key environmental information.
4. Simulated Environment Evaluation: Tested system performance in real-world scenarios (e.g., urban and rural settings) and compared results with a baseline audio system.

Research Outcomes

Specific Results

• PathFinder significantly reduced users' cognitive load while maintaining high perceived safety in both complex and simple scenarios.
• The multimodal interface enabled users to create "mental maps," providing a comprehensive understanding of their surroundings, particularly in identifying dynamic and static obstacles.
• Users preferred the combination of tactile and audio feedback, with tactile offering an overview and audio supplementing critical details.

Advantages Over Existing Solutions

• Compared to systems relying solely on audio feedback, PathFinder mitigated safety issues caused by insufficient information delivery in complex scenarios.
• PathFinder actively adapts to users' personalized needs, such as adjusting the level of detail based on visual acuity.
• The combination of audio and tactile feedback significantly enhanced users' spatial awareness and sense of control over their environment.

Experimental or Evaluation Results

• In a three-factor experiment involving 16 participants, PathFinder reduced average cognitive load (7.75 vs. 11.09) and maintained high consistency in perceived safety across urban and rural scenarios.
• Qualitative feedback indicated that users generally preferred the multimodal interface, particularly for dynamic obstacle detection and environmental information delivery.
• Users found the physical feedback from tactile buttons more reliable and accessible than systems relying solely on audio information.

Limitations and Future Directions

• Limitations:
  • Small sample size (N=5 for workshops and N=16 for experiments), which may introduce selection bias.
  • Experiments were conducted in simulated environments, not real autonomous vehicle settings. Additionally, extreme weather conditions were not considered.
  • PathFinder currently does not cover the entire journey and needs integration with systems for other navigation stages (e.g., locating autonomous vehicles).
• Future Directions:
  • Test PathFinder's performance in real vehicle environments to enhance external validity.
  • Explore the possibility of integrating multimodal systems (e.g., handheld devices combined with tactile navigation bars) to improve accessibility throughout the journey.
  • Optimize hardware design (e.g., miniaturization or integration with vehicle interior components) to enhance portability and applicability.

This study demonstrates a profound focus on the independence and travel safety of BVIPs through the design and validation of a multimodal interface. PathFinder showcases the innovative potential of leveraging tactile, auditory, and visual technologies to address practical challenges, while highlighting the importance of structural integration and personalized design.