Dynamic Field of View Restriction in 360º Video: Aligning Optical Flow and Visual SLAM to Mitigate VIMS

Title of the Paper

Dynamic Field of View Restriction in 360° Video: Aligning Optical Flow and Visual SLAM to Mitigate VIMS

Paper Information

• Subject Area: Virtual Reality (VR), User Experience, Visual-Induced Motion Sickness (VIMS) Mitigation
• Keywords: Visual-Induced Motion Sickness (VIMS), 360° Video, Optical Flow, Visual SLAM, Field of View (FoV) Restriction, Virtual Reality (VR), User Studies

Research Background and Issues

• Key Problems or Challenges Identified:

  1. With the increasing popularity of head-mounted display (HMD)-based virtual reality, many users experience visually induced motion sickness (VIMS), which limits user experience and hinders the widespread adoption of VR.
  2. Existing studies have shown that restricting the field of view (FoV) can alleviate VIMS, but this comes at the cost of reduced user immersion.
  3. For pre-rendered formats like 360° video, current VIMS mitigation methods (e.g., FoV restriction) are not yet compatible.

• Significance: VIMS affects up to 67% of adult users (with women being more susceptible), causing symptoms such as nausea, dizziness, and discomfort. This significantly limits the potential and widespread adoption of VR technology.

• Research Motivation and Related Work: The authors aim to explore how to dynamically adjust FoV in 360° video to reduce VIMS caused by fast-moving scenes while preserving user immersion as much as possible. Previous studies have attempted to use optical flow algorithms and SLAM technology but lack a comprehensive and systematic approach for application in 360° video.

Solution

• Proposed Method or Solution: The authors propose a dynamic FoV restriction technique that integrates the following two types of information:

  1. Visual SLAM (Simultaneous Localization and Mapping) technology to identify the camera motion state (stationary or moving) in the video.
  2. Optical Flow analysis to capture motion patterns in the user's peripheral vision and dynamically adjust the FoV.

• Innovations:

  1. This is the first integration of visual SLAM and peripheral optical flow data in 360° video to achieve dynamic FoV adjustment.
  2. Unlike other studies, this method triggers FoV restriction only in VIMS-inducing scenarios, minimizing interference with the user experience.
  3. The system design is adaptable to various video scenarios (e.g., stationary shots, slow-moving scenes, and fast-moving scenes).

• Implementation Steps and Key Techniques:

  1. Optical Flow Calculation:
     • Preprocess the video to reduce the computational load of optical flow, using the dense Gunnar Farnebäck algorithm to calculate per-pixel optical flow intensity.
     • Grid and downsample the optical flow data, recording it in a CSV file.
  2. Visual SLAM:
     • Use OpenVSLAM to track and localize the 360° video.
     • Generate motion classifications ("stationary," "moving") through feature extraction.
  3. FoV Restriction Implementation:
     • Implement dynamic FoV restriction in Unity, dividing the display into a central display area, a transition area, and a peripheral occlusion area.
     • Adjust the FoV size (dynamically varying between 40° and 90°) using optical flow and motion classification data, with smooth transitions to reduce abruptness perceived by users.

Research Outcomes

• Specific Results:

  1. The dynamic FoV approach effectively reduces VIMS symptoms. Compared to fixed FoV strategies, dynamic FoV reduces VIMS while maintaining higher immersion.
  2. Users clearly benefit from the VIMS mitigation effect of dynamic FoV in "fast-moving" scenes.

• Advantages Over Existing Solutions:

  1. Dynamic FoV triggers FoV restriction only when necessary, reducing visual interference compared to fixed FoV strategies.
  2. The system design ensures that 360° video retains a high level of user immersion, allowing users to freely explore appropriate scenes.

• Experiments and Evaluation Results:

  • A user study with 23 participants showed:
    • In fast-moving scenes, dynamic FoV restriction performed comparably to fixed FoV restriction in alleviating VIMS.
    • Compared to fixed FoV, dynamic FoV demonstrated significant advantages in maintaining immersion.
    • Semi-structured interviews revealed that most users accepted the dynamic FoV adjustments and perceived them as part of the video narrative, though some users found the frequent changes confusing.

• Limitations and Future Directions:

  1. Study Limitations:
     • Only one commercial 360° video was tested. While the video featured diverse scenes, the results may not be fully generalizable to other types of videos.
     • The system currently reacts only to content optical flow and does not integrate user body motion data.
  2. Future Directions:
     • Extend the method to longer videos and different types of content.
     • Optimize the dynamic FoV response strategy, such as adapting to "slow-moving" scenes to reduce visual instability.
     • Incorporate user behavior (e.g., head movement data) to further personalize the dynamic FoV strategy.

Conclusion

This paper presents a novel VIMS mitigation solution for 360° video by integrating visual SLAM technology and peripheral optical flow data to dynamically adjust the FoV, balancing VIMS relief and immersion. Dynamic FoV received high acceptance in user studies and demonstrated effective VIMS mitigation, providing valuable insights for future design and optimization of 360° video user experiences.