PassengXR: A Low Cost Platform for Any-Car, Multi-User, Motion-Based Passenger XR Experiences

Document Title

PassengXR: A Low Cost Platform for Any-Car, Multi-User, Motion-Based Passenger XR Experiences

Document Information

• Subject Area: Virtual Reality (VR), Augmented Reality (AR), Extended Reality (XR), In-Vehicle Technology
• Keywords: Mixed Reality, Extended Reality, In-Vehicle Applications, Toolkit, Multi-User, Reference Framework, Position Tracking, Unity Development

Research Background and Problem

• Problems or Challenges:

  • Traditional in-vehicle XR systems face high development costs (e.g., driving simulators or equipment procurement), tracking stability issues, and practical difficulties in real-time design testing. Moreover, existing systems often require constrained experimental environments, which differ significantly from real driving scenarios.
  • Most current platforms are limited to PC VR devices, which are expensive, and most headsets experience drift issues in dynamic vehicle environments.

• Research Importance:

  • With the rise of autonomous vehicles, passenger travel time can be utilized for leisure, entertainment, or productivity activities, making the design of low-cost and highly mobile in-vehicle XR essential. For researchers and developers, a solution compatible with more vehicles and offering high portability is critical to advancing in-vehicle XR technology.

• Research Motivation and Related Work:

  • Studies on in-vehicle XR experiences, such as CarVR and Holoride, have demonstrated preliminary applications based on vehicle motion and sensor data but face several limitations: limited device support, poor compatibility, and lack of multi-user functionality.
  • Therefore, it is necessary to innovate a technical platform that is widely compatible with different vehicle models and usage scenarios while reducing technical barriers and hardware costs.

Solution

• Method or Solution:

  • Introduce PassengXR, an open-source and low-cost platform that allows designers to develop multi-user in-vehicle XR experiences based on Unity.
  • Key technologies include IMU (Inertial Measurement Unit) sensors, OBD-II vehicle ports, GNSS/GPS, and wireless transmission of live or recorded vehicle data.

• Innovations:

  • Enable multi-user real-time sensing and playback of vehicle motion data (IMU orientation, OBD-II speed, and global positioning).
  • Include built-in tools for headset alignment and correction to address drift issues.
  • Provide Unity configuration support for any vehicle and multi-user functionality.
  • Offer laboratory recording and playback tools, allowing researchers to develop based on real driving data without requiring a physical vehicle.

• Implementation Steps and Key Technologies:

  1. Hardware Design:

     • Use Arduino ESP32 modules, IMU, OBD-II, and GNSS sensors to collect and broadcast vehicle motion data.
     • Client devices include Pico Neo 3 Pro headsets, receiving vehicle data via USB or Wi-Fi.

  2. Software Development:

     • Utilize the Unity Motion Platform framework combined with Protobuf protocol to parse sensor data.
     • Provide various solutions for vehicle-headset alignment and drift correction.

  3. Core Features:

     • Support dynamic alignment, IMU drift correction, and multi-user shared in-vehicle experiences.
     • Support recording and playback of real driving data to achieve "immersive development simulation."

  4. Compatibility Enhancement:

     • Unity plugins compatible with various headset devices, such as Pico, HTC Vive, and even open AR device integration.

Research Outcomes

• Specific Results:

  1. Successfully developed and deployed a low-cost in-vehicle XR platform, with hardware costs ranging from $480 to $1000 (significantly lower than most market solutions).
  2. Provided three dedicated developer scenarios, including entertainment, information workspace, and multi-user environments.
  3. Offered three sets of real driving record data for future experiments and design use.

• Advantages Compared to Existing Solutions:

  • Supports any in-vehicle XR headset device, overcoming the dependency of commercial services like Holoride on specific vehicle models and devices.
  • Provides IMU drift dynamic correction tools and a developer-friendly toolchain, offering stronger sharing and openness.

• Experimental or Evaluation Results:

  • Tested drift issues on multiple headset devices, with Pico Neo 3 Pro showing the best performance, minimal drift (~7.3°~22 minutes).
  • Successfully implemented multi-user collaboration, dynamic environment binding, and real-time vehicle content recording and playback.

• Limitations and Future Directions:

  • Currently supports only 3DoF headset tracking; high-end 6DoF devices like Meta Quest still face tracking drift issues.
  • OBD-II speed sampling frequency is constrained by vehicle models, with older vehicles potentially experiencing sensor issues.
  • Comprehensive user experience testing has not yet been conducted; future versions should enhance usability evaluations for designers and users.
  • Future expansions should consider support for LiDAR, AR glasses, and other smart devices, as well as compatibility development with public transportation systems.

Conclusion

PassengXR lowers the barrier for in-vehicle XR development through groundbreaking design. Its open-source and modular strategy will contribute to future research and implementation of autonomous driving and in-vehicle interaction interfaces. In the long term, standardization, ecosystem extensibility, and improvements in 6DoF support remain critical goals.