VRhook: A Data Collection Tool for VR Motion Sickness Research

Title of the Paper

VRhook: A Data Collection Tool for VR Motion Sickness Research

Bibliographic Information

• Subject Area: Virtual Reality (VR), Motion Sickness, Machine Learning, Data Collection
• Keywords: Automated Data Collection, Motion Sickness, Virtual Reality, Machine Learning, Dataset Generation, Dynamic Hooking, Self-Reported Comfort, Graphics Rendering, Risk Factor Detection

Research Background and Problem Statement

• Identified Problems or Challenges:

  • The rapid development of VR technology is hindered by motion sickness, which affects approximately 40% of users with symptoms such as fatigue, dizziness, and nausea, impacting their experience.
  • Current comfort ratings for VR games largely rely on manual expert analysis, which is costly and difficult to scale with the growing gaming industry.
  • Machine learning methods have the potential to improve VR motion sickness detection, but obtaining large-scale labeled data remains a challenge.

• Why This Problem Is Important:

  • VR motion sickness not only significantly affects user experience but also poses a potential barrier to the further development of the VR industry.
  • Providing an automated, scalable data collection solution would greatly facilitate related machine learning research and applications.

• Research Motivation and Related Work:

  • While some studies have attempted to use machine learning models to detect motion sickness, training these models still requires large amounts of labeled data.
  • Traditional data acquisition methods (manual labeling or custom-built scenarios) have limitations, such as being labor-intensive and lacking generalizability.
  • This paper proposes an automated data collection tool that does not require access to game source code to address these challenges.

Solution

• Proposed Method or Solution:

  • Developed a data collection tool called VRhook, capable of extracting labeled data from any VR game without requiring access to its source code.
  • Utilized "Dynamic Hooking" technology to inject custom code into runtime memory, capturing video frames and associated transformation matrices from games.

• Innovations:

  • Enabled source-code-independent data collection for commercial VR games.
  • Automated the extraction of labeled data (e.g., rotation, velocity, acceleration) to support machine learning model training.
  • Supported in-game overlay interfaces for collecting user self-reported comfort ratings.

• Implementation Steps and Key Technologies:

  1. Hooking Technology: Inserted code at fixed addresses in low-level operating system graphics stacks (e.g., DirectX or OpenGL) to capture standardized graphical data.
  2. Matrix and Label Generation: Used captured matrices to generate key action labels, including camera velocity, acceleration, multi-axis rotation, and field of view.
  3. Video Frame Capture: Intercepted rendered frames through the GPU's Swap Chain mechanism and designed an efficient memory buffering strategy to avoid performance bottlenecks.
  4. User Rating Collection: Integrated feedback interfaces (e.g., Microsoft Surface Dial) to record real-time user-perceived motion sickness comfort levels.
  5. Machine Learning Model Support: Generated datasets for training machine learning models to detect risk factors and predict comfort levels.

Research Outcomes

• Specific Achievements:

  • Developed a data collection tool that successfully extracted labeled data from 127 popular VR games on Steam.
  • Built a two-stage machine learning pipeline to detect VR game risk factors and predict user comfort ratings.
  • Validated the tool's application in roller coaster-style games, generating large-scale datasets that supported accurate risk factor predictions.

• Comparison with Existing Solutions:

  • Compared to manual labeling or custom-built game data collection, VRhook offers significant advantages in automation and scalability.
  • The use of dynamic hooking technology and content-based label generation methods improved data quality and generalizability.

• Experimental or Evaluation Results:

  • The first-stage risk factor detection model achieved an accuracy of 76%-78% and an F1 score of 71%-76%.
  • The second-stage comfort rating prediction model, based on a support vector machine, achieved an accuracy of 70% and a weighted F1 score of 69%.

• Limitations and Future Directions:

  • The current study validated the tool with a limited number of games and participants; further data collection and model optimization are needed.
  • Some less-studied risk factors (e.g., sound effects, graphical realism) were not covered, requiring future research.
  • Plans to modify the tool to support emerging VR runtime platforms like OpenXR and expand its capabilities to capture physiological data (e.g., heart rate and electrical signals).

Summary and Outlook

VRhook provides a revolutionary data collection solution to address the lack of data in VR motion sickness research. Its automated data labeling and user rating functionalities open up numerous opportunities for machine learning research and can support risk assessment and game design optimization in the VR gaming industry. Future research goals include expanding the data scale, exploring more machine learning architectures, and extending the tool's functionality to multi-platform environments and multidimensional data collection.