"It Must Be Gesturing Towards Me": Gesture-Based Interaction between Autonomous Vehicles and Pedestrians

Title of the Paper

“It Must Be Gesturing Towards Me": Gesture-Based Interaction between Autonomous Vehicles and Pedestrians

Paper Information

• Research Domain: Human-Computer Interaction, focusing on interaction design between autonomous vehicles and pedestrians
• Keywords: Autonomous vehicle-pedestrian interaction, gesture-based interaction, autonomous driving, external human-machine interface (eHMI), pedestrian safety

Research Background and Problem

• Problem Description: Current autonomous vehicles lack effective communication mechanisms, leading to unclear or failed interactions with pedestrians, which can result in traffic safety and efficiency issues. For example, the “Frozen Robot Problem” describes the inability of autonomous vehicles to coordinate adequately with pedestrians, potentially causing traffic chaos.

• Importance: The understanding and interaction between pedestrians and autonomous vehicles directly affect traffic safety and flow. Therefore, developing an interface capable of accurately conveying vehicle intentions is crucial.

• Research Motivation: Based on existing studies, gestures are commonly used for interaction between pedestrians and traditional drivers but are rarely applied in the interaction design of autonomous vehicles. This study attempts to introduce gesture design as an external human-machine interface (eHMI) and explore whether it performs better than current mainstream eHMI designs (e.g., lights or text).

• Research Questions:

  1. What are the differences in effectiveness between gesture-based interaction and other eHMI designs? Which gestures are most effective?
  2. What are the potential factors contributing to these differences?

Solution

• Methods or Solutions:

  • This study designed a gesture-based eHMI, selecting eight gestures (including four gestures indicating “yielding” intentions and four indicating “non-yielding” intentions).
  • Virtual reality (VR) experiments and online surveys were used to validate the effectiveness of these gestures.

• Innovations:

  • Proposed an eHMI concept inspired by everyday interactions between drivers and pedestrians.
  • Developed a gesture set based on common gestures in daily traffic scenarios, screened for semantics, cultural adaptability, and clarity to select the best gestures.
  • Provided empirical analysis of gesture effectiveness and design improvements through VR and online surveys.

• Implementation Steps:

  1. Gesture Selection:
     • Created a gesture library, including everyday gestures, sport-specific movements, traffic control signals, and sign language.
     • Screened gestures through an expert panel, categorizing them by semantics, excluding gestures with significant cultural differences, and optimizing clarity.
     • Ultimately selected four “yielding” gestures and four “non-yielding” gestures.
  2. Experiment Design:
     • Used VR technology to construct virtual scenarios simulating pedestrian-autonomous vehicle interactions at uncontrolled intersections.
     • Participants controlled pedestrian behavior to respond to vehicles with different eHMI designs.
  3. Data Collection and Analysis:
     • Collected multidimensional data, including error rates, observation duration, hesitation duration, and physiological indicators (e.g., heart rate, skin conductance).
     • Analyzed participants’ difficulty in understanding, perceived danger, and overall experience with different designs.
  4. Online Survey:
     • Used the AHP method to evaluate the clarity, familiarity, and politeness of gestures, analyzing respondents’ ratings for each gesture.

Research Findings

• Specific Results:

  • VR Experiment Findings:
    • Gesture-based eHMI (especially certain gestures like Y3 and N1) outperformed existing designs in reducing pedestrians’ observation and hesitation duration.
    • Some gestures (e.g., Y2, Y3) significantly enhanced pedestrians’ sense of safety.
  • Survey Results:
    • Gestures rated as “clear” and “familiar,” such as Y2 and N1, received the highest evaluations, while Y3 performed best in terms of “politeness.”
    • Age groups showed differences in preferences: younger participants prioritized familiarity, while older participants valued politeness more.

• Advantages Comparison: Compared to traditional eHMI designs relying solely on lights, gesture designs are more expressive, not only conveying vehicle intentions but also imbuing vehicles with emotional and human-like characteristics, thereby enhancing pedestrians’ trust and acceptance of autonomous vehicles.

• Limitations and Future Directions:

  • Current experimental scenarios are relatively simple, simulating only pedestrian-autonomous vehicle interactions. Future studies should expand to mixed traffic environments (e.g., pedestrians, bicycles, and other vehicles).
  • The visibility limitations in VR environments do not reflect real-world conditions. Subsequent research should optimize gesture designs to ensure recognizability at greater distances.
  • Further experiments could explore the combined effects of light and gesture-based interaction designs.
  • This study was primarily conducted in China; future research could expand globally to investigate the impact of cultural differences on gesture interpretation.

Conclusion

This study explored efficient communication methods between autonomous vehicles and pedestrians through gesture-based interaction design. VR experiments and online surveys demonstrated that certain gestures significantly improve interaction experiences while highlighting the importance of gesture clarity, cultural differences, and learning costs. Future designs are recommended to combine multiple interaction modalities and be validated in more complex traffic environments.