XR-OOM: MiXed Reality driving simulation with real cars for research and design

Document Title

XR-OOM: MiXed Reality driving simulation with real cars for research and design

Document Information

• Subject Area: Mixed Reality (XR) driving simulation, integrating virtual reality and real vehicles for design and research purposes.
• Keywords: XR, Mixed Reality, Driving Simulation, Cars, User Research, Design, Safety, Behavioral Studies

Research Background and Issues

• Identified Problems or Challenges:

  1. Current driving simulators exist in various forms, but experimental outcomes (e.g., realism) are limited—for instance, fixed-base simulators in labs lack authentic driving experiences.
  2. Mixed Reality (XR) technology has been applied to driving simulation, but its experimental methods and safety require further validation.
  3. When developing vehicle interfaces and autonomous driving systems, there is a challenge in balancing design and testing environments: how to enable rapid prototyping while retaining user behavior data close to real-world conditions.

• Research Significance: High-fidelity mixed simulation systems that simulate real driving experiences can maximize experimental realism, aiding the development and evaluation of next-generation in-car interfaces and driving assistance technologies.

• Research Motivation and Related Work:

  • Mixed Reality technologies (e.g., MAXIM and VR-OOM) have demonstrated potential, but stricter safety and effectiveness validations are lacking.
  • To meet the requirements of driving simulation research for safety, effectiveness, and repeatability, new Mixed Reality methods need to be developed and validated.

Solution

• Proposed Method or Solution:

  • Develop the XR-OOM system, based on the Varjo XR-1 headset and Unity simulation environment, overlaying virtual scenes onto real driving environments.
  • The system utilizes "video pass-through" functionality to combine real-world environments with virtual objects for driving experiments.

• Innovative Features:

  1. The system integrates high-fidelity Mixed Reality with real vehicle driving tests.
  2. It is the first to rigorously validate the safety and effectiveness of an XR Mixed Reality system in real driving environments.
  3. Provides standardized experimental procedures to ensure reliability and repeatability in basic driving task tests.

• Implementation Steps and Key Technologies:

  1. Hardware Setup:
     • XR-1 headset for video and virtual object overlay.
     • Cameras for environmental capture and position integration.
  2. Software Development:
     • Designed rendering programs to integrate virtual objects into real-world scenes.
     • Used Simultaneous Localization and Mapping (SLAM) technology for environmental positioning.
  3. Research Instruments:
     • The system is equipped with eye trackers, 360° cameras, and external cameras for data recording.
  4. Experiment and Validation Design:
     • Validated system performance in cockpit tasks and low-speed driving tasks.
     • Compared different experimental conditions (no headset, headset without virtual objects, headset with virtual objects).

Research Outcomes

• Specific Results:

  1. Achieved hardware and software integration of the XR-OOM system.
  2. Validated the system's safety and effectiveness in cockpit tasks (e.g., using windshield wipers, adjusting seats) and low-speed driving tasks.
  3. Experimental results confirmed the realism of virtual objects, prompting drivers to take the virtual environment seriously.

• Advantages Compared to Existing Solutions:

  1. Provides a highly immersive driving experience, offering greater realism compared to traditional lab-based or purely VR simulators.
  2. Enables researchers to simulate complex driving scenarios that are difficult to test in real environments (e.g., obstacles, visual distractions).
  3. Supports extensive research on driving environments and driver assistance technologies.

• Experimental or Evaluation Results:

  1. Participants successfully completed cockpit tasks, although task difficulty (e.g., reading small instrument details) was affected by the display resolution of the device.
  2. Driving trajectory differences were minimal, indicating the system did not significantly impact driving behavior.
  3. Participants reported mild discomfort during the experience, such as neck fatigue caused by headset weight.

• Limitations and Future Directions:

  1. Limitations:
     • Current validation focuses on low-speed driving tasks; high-speed driving experiments require further testing.
     • Reverse driving and complex driving tasks have not been thoroughly tested.
     • Mixed Reality headsets have limitations in dynamic light adjustment and high dynamic range performance.
  2. Future Directions:
     • Improve system hardware, particularly headset weight and camera performance.
     • Further test safety in high-dynamic scenarios, such as simulating urban traffic environments.
     • Expand participant samples to enhance research generalizability.

Conclusion

The XR-OOM system proposed by the authors combines the advantages of virtual and real driving simulations and is the first to validate its potential for experimental research in terms of safety, repeatability, and practicality. The research results lay a critical foundation for further applications of Mixed Reality driving simulation in in-car interface design and autonomous driving research.