Bumpy Ride? Understanding the Effects of External Forces on Spatial Interactions in Moving Vehicles

Research Background and Issues

Issues and Challenges

• This paper focuses on the impact of vehicle motion (e.g., G-forces, vibrations, and irregular driving trajectories) on user interaction performance when using head-mounted displays (HMDs) for spatial interaction in mobile automotive environments.
• The authors point out that the dynamic behavior of vehicles significantly reduces interaction accuracy, increases task completion time, and heightens workload, potentially hindering the efficient use of HMDs.

Significance

• With the proliferation of autonomous vehicles, passengers are increasingly inclined to use HMDs for work, entertainment, or other non-driving-related tasks. However, the influence of dynamic vehicle factors on HMD interaction methods could become a major obstacle to the practical use of these devices.
• Existing research predominantly relies on low-fidelity driving simulators, lacking systematic analysis of HMD interaction performance under real-world driving conditions.

Research Motivation and Related Work

• Motivation: Previous studies have explored HMD interaction methods in static environments, but solutions for addressing complex in-vehicle motion environments remain insufficient.
• The authors reference prior research on in-vehicle interaction methods such as touchscreens, eye tracking, and gesture interaction, as well as foundational work on the effects of motion using simulators.

Proposed Solution

Proposed Method

• This study investigates the performance of four interaction methods (Gaze&Pinch, DirectTouch, Handray, and HeadGaze) in a real vehicle environment.
• Participants completed Fitts’ law-based interaction tasks under two conditions (stationary and moving vehicle) to optimize and compare the operational effectiveness of different interaction methods.

Innovations in the Method

1. Systematically incorporates road surface types (smooth, mixed, bumpy) and curve path types (short curves, long curves, straight paths) as influencing factors.
2. Provides quantitative data and qualitative feedback on the accuracy, workload, and usability of HMD interaction devices in dynamic environments.
3. Refines the factors affecting interaction performance, such as selection offset and task completion time during vehicle motion.
4. Proposes interaction design recommendations for dynamic environments, offering guidance for future HMD development.

Implementation Steps

1. Experimental Design: Comparative tests were conducted with 24 participants. Interaction tasks involved selecting targets in a virtual HMD environment using four different methods.
2. Data Collection: Real-time recording of in-vehicle sensor data (e.g., IMU acceleration) and user interaction motion trajectories (e.g., hand and head movements).
3. Evaluation Metrics: User subjective workload was measured using NASA-RTLX, task efficiency was calculated using Fitts’ law, and error rates and selection accuracy were assessed.

Research Findings

Key Findings

1. Task Performance: Vehicle motion significantly reduced the accuracy and efficiency of all interaction methods, increasing task error rates. Handray performed best under dynamic conditions, while Gaze&Pinch was most affected by motion.
2. Workload and Usability: Overall workload increased significantly during vehicle motion, with HeadGaze and Gaze&Pinch being the most affected. Usability scores were higher (good level) under stationary conditions but dropped to average under dynamic conditions.
3. Subjective Feedback: Handray was considered the best interaction method in dynamic contexts, followed by DirectTouch. Gaze&Pinch was deemed most suitable for privacy-sensitive scenarios but requires further technical optimization.

Experimental or Evaluation Results

• Error Rate: Gaze&Pinch had the highest average error rate under dynamic conditions (36%), while Handray had the lowest (20.4%), demonstrating greater interaction robustness.
• Selection Accuracy: DirectTouch and Handray exhibited the smallest selection deviations in dynamic environments, making them more suitable for such conditions.
• Task Time: Selection times increased significantly for all methods under dynamic conditions, though Gaze&Pinch and DirectTouch showed shorter times under certain road conditions.

Comparative Advantages Over Existing Solutions

• This study is the first to combine the effects of various real-world dynamic driving conditions with interaction methods, providing critical data to support the in-vehicle application of future AR/VR devices.

Limitations and Future Directions

• Limitations:
  1. The weight of the HMD affected the accuracy of eye tracking and head movements.
  2. The hand tracking system lacked stability in strong light environments.
  3. Most participants were automotive industry employees, which may introduce bias.
• Future Directions:
  1. Introduce lighter, higher-performance HMDs and tracking devices.
  2. Expand research to include different driving speeds and stress-testing scenarios.
  3. Develop algorithmic optimizations, such as motion compensation and target selection prediction mechanisms.

Conclusion

This study thoroughly explores the impact of vehicle motion on HMD interaction performance. Through field experiments and data analysis, it not only validates the primary influencing factors but also proposes specific design recommendations, laying a solid foundation for immersive experiences in future autonomous vehicles. Additionally, it provides valuable practical insights for mixed reality interaction design.