The Halting problem: Video analysis of self-driving cars in traffic

Title of the Paper

The Halting Problem: Video Analysis of Self-driving Cars in Traffic

Paper Information

• Subject Area: Interaction between self-driving cars and traffic
• Keywords: self-driving cars, traffic interaction, social behavior, HCI, human-computer interaction, road users, traffic safety, behavior analysis, video analysis, collaborative driving

Research Background and Problem

• Identified Problems or Challenges:

  • Self-driving cars still struggle to effectively handle road interactions in socialized traffic environments, such as "yielding" and "going," when interacting with other road users.
  • The actual performance of current self-driving systems remains proprietary information, and the public primarily learns about their development through videos shared by ordinary users.
  • Self-driving cars need to clearly communicate their intentions in complex environments involving various road users, such as pedestrians and vehicles.

• Significance of the Research:

  • Traffic is a highly complex social domain, and the safe operation of self-driving systems requires seamless interaction with other road users.
  • Failures or ambiguities in traffic interactions can lead to dangerous behaviors or potential collisions. Exploring how to design self-driving systems that can collaborate effectively with human users is critical for traffic safety.

• Motivation and Related Work:

  • Despite rapid advancements in self-driving technology, challenges remain in handling social interactions and basic tasks, such as correctly interpreting traffic signals or pedestrian intentions.
  • Existing research often focuses on explicit external interaction methods (e.g., lights, signals), but there is insufficient study on implicit interactions (e.g., movement trajectories, speed intentions).
  • By analyzing publicly available third-party videos on the internet, this study aims to uncover the challenges faced by self-driving cars in real-world traffic environments.

Solution

• Proposed Method or Solution:

  • Analyze road test videos of self-driving cars (Waymo and Tesla FSD) uploaded by third parties on YouTube to examine the interaction behaviors of these cars with pedestrians and other vehicles.
  • Use "yielding," a simple yet widely occurring form of road interaction, as the research basis to conduct an in-depth analysis of yielding behaviors through video data.

• Innovative Aspects:

  • Utilize publicly available video data to analyze the real-world road behavior of self-driving cars, rather than relying on laboratory tests or proprietary company data.
  • Introduce the analysis of the dynamics and sequential nature of social interactions into human-computer interaction (HCI) research, discussing how self-driving systems adapt to these human behaviors.

• Implementation Steps and Techniques:

  • Collect a large number of YouTube videos (over 16 hours each for Waymo and Tesla FSD).
  • Systematically observe the videos and select segments related to "yielding," focusing on problem scenarios, smooth interactions, and abnormal events.
  • Conduct multimodal analysis of actions using theories of action sequences and interaction from human sociology.

Research Findings

• Specific Findings:

  • Classified and analyzed typical (successful or failed) cases of "yielding" by self-driving cars:
    • Case 1: Waymo failed to correctly respond to a pedestrian's hand gesture for yielding, causing confusion in traffic interaction.
    • Case 2: Waymo successfully yielded at an intersection by waiting for other vehicles to proceed through specific stopping behavior.
    • Case 3: Tesla FSD exhibited a complex three-way dynamic interaction involving driver intervention and a pedestrian.
    • Case 4: Tesla failed to "claim the right of way" at a four-way stop, repeatedly stopping and starting, creating uncertainty for other drivers.
  • Found that the success or failure of traffic interactions involves multimodal communication (e.g., gestures, vehicle speed changes) and mutual adjustment of interaction timing.

• Advantages Compared to Existing Solutions:

  • Traditional research often relies on laboratory and pre-set scenarios, which fail to capture the complexity of real-world interactions. This study reveals the actual complexities of traffic through the analysis of publicly available videos.
  • Proposes traffic interaction as a new form of machine/human interaction, emphasizing its deep social collaboration attributes.

• Experimental or Evaluation Results:

  • The study shows that Waymo and Tesla FSD have not yet fully developed the ability to respond to human social interactions in complex yielding scenarios:
    • Waymo sometimes exhibits passive "waiting" behavior, which may be a programmed mode rather than genuine "social understanding."
    • Tesla struggles with handling dynamic timing and sequencing, requiring human intervention.

• Limitations and Future Directions:

  • The study is based on a limited video dataset and may be subject to selection bias.
  • Internal system data (e.g., Waymo's programming code) is unavailable, making it impossible to verify specific decision-making mechanisms.
  • Future research should focus on designing self-driving algorithms capable of effective social interaction, such as better understanding human intentions through speed adjustments and interaction timing.
  • Explore how explainable design can help other road users understand the intentions of self-driving cars, reducing the burden on road users.