Portobello: Extending Driving Simulation from the Lab to the Road

Title of the Paper

Portobello: Extending Driving Simulation from the Lab to the Road

Paper Information

• Subject Area: Human-Computer Interaction, Driving Simulation, Mixed Reality (XR)
• Keywords: Driving Simulation, Human-Computer Interaction, Mixed Reality, Autonomous Driving, Platform Portability, Ecological Validity, User Studies, Cross-Platform Experiments, Virtual Reality, XR-OOM

Research Background and Problems

• Identified Problems or Challenges:

  • In autonomous driving research, traditional experiments typically begin with lab-based driving simulation and gradually transition to real-world road testing to mitigate risks. However, replicating experimental designs between the physical and virtual worlds is challenging.
  • Laboratory driving simulators allow precise control of experimental environments but cannot fully replicate real-world vehicle dynamics and inertial forces, which impacts immersion and ecological validity.
  • Current road-based driving simulators lack the ability to accurately design and manage virtual objects in the external environment, limiting experimental diversity.

• Significance of the Research:

  • Improving the ecological validity of driving simulation experiments is a critical challenge in the fields of autonomous driving and human-computer interaction. Addressing platform portability issues can help researchers more effectively validate the real-world applicability of lab-based simulation studies.
  • This research contributes to enhancing the development and testing capabilities of driving simulation technology, better adapting to real-world road dynamics, and advancing the study of safe driving systems.

• Motivation and Related Work:

  • Existing research employs XR technologies to enhance the perceptual immersion of driving simulations, such as mixed reality systems based on visual SLAM or 360° cameras, but lacks unified platform portability capabilities.
  • Wynne et al. found that most driving simulation studies lack validation research, leading to uncertainties between lab-based simulations and real-world driving outcomes.
  • The authors aim to address the challenges of cross-platform experimentation by achieving "platform portability" in experimental design, thereby advancing ecological validity research in driving simulation.

Solution

• Proposed Solution:

  • Develop a driving simulation infrastructure called "Portobello," which leverages localization technologies and software from robotics to enable the transfer of experimental designs between lab-based simulators and road-based driving simulators.
  • Integrate Portobello into the existing road-based driving simulation system XR-OOM to enable location-based event design and rendering, supporting twin experiments across environments (lab and real-world roads).

• Innovations:

  • Achieved "platform portability" in driving simulation experimental design for the first time. By associating virtual objects with a shared map, virtual events are precisely aligned with real-world road environments.
  • Replaced GPS with LiDAR sensors to achieve higher localization accuracy, suitable for urban environments.
  • Provided a unified framework for experimental design and event scheduling, allowing researchers to design experiments without requiring in-depth knowledge of underlying localization technologies.

• Implementation Steps and Techniques:

  • Map Generation: Use LiDAR devices to scan the study area and create precise point cloud maps as the basis for experimental design.
  • Event Design: Drag and drop virtual objects onto the map and control event timing and location using collision triggers.
  • Platform Integration: Achieve real-time data communication between ROS and Unity to dynamically synchronize vehicle positions with the virtual environment.
  • Experiment Execution: Conduct twin experiments using a fixed lab-based driving simulator and a road-based driving simulator powered by the Portobello system to validate platform portability.

Research Outcomes

• Specific Results:

  • Successfully conducted cross-environment twin experiments, validating the "platform portability" of the Portobello system.
  • Compared results from lab-based and road-based simulators, revealing significant differences in participant behavior, immersion, and user experience.
  • Demonstrated the advantages of different simulation platforms: labs are better suited for quantitative analysis, while real-world roads better reflect natural behaviors in complex environments.

• Advantages:

  • Enables researchers to seamlessly transition from lab experiments to real-world road experiments, reducing redundant experimental design efforts.
  • High-precision maps and location synchronization technology based on LiDAR address the localization errors of traditional GPS systems.
  • Enhances the ecological validity and real-world applicability of research.

• Experimental or Evaluation Results:

  • Conducted twin "pedestrian crossing cooperation" experiments with 32 participants across both platforms. Results showed that participants exhibited more natural head movements and considered decision-making as more critical in the real-world simulator.
  • Survey assessments revealed that the lab-based simulator scored higher in safety and comfort, while the road-based simulator offered greater naturalness and realism.

• Limitations and Future Directions:

  • The dynamic occlusion of virtual objects remains unresolved, potentially affecting the immersion of the XR experience.
  • Participants frequently complained about the weight and field-of-view limitations of head-mounted devices; future technological advancements should improve XR hardware comfort.
  • Random event interference affected experimental consistency; more refined designs could mitigate such impacts in the future.
  • To enhance the interaction between virtual events and real-world scenes, integrating real-time dynamic scanning technologies is recommended.

Open Science and Contributions

• Open Resources: The source code for Portobello is publicly available on GitHub.
• Research Support: This study was funded by the Toyota Research Institute and Woven by Toyota.

Through this research, the Portobello system provides innovative solutions for scientific and industrial applications in the field of driving simulation research and demonstrates the potential of open science.