RadarVR: Exploring Spatiotemporal Visual Guidance in Cinematic VR

Title of the Paper

RadarVR: Exploring Spatiotemporal Visual Guidance in Cinematic VR

Paper Information

• Subject Area: Virtual Reality (VR), User Interface Interaction, Visual Guidance
• Keywords: Virtual Reality, 360-degree Video, Cinematic Virtual Reality, Spatiotemporal Visual Guidance, RadarVR, Human-Computer Interaction

Research Background and Problem

• Problems or Challenges:

  1. In Cinematic Virtual Reality (CVR), viewers may miss important events in the scene (Regions of Interest, ROI) due to the limitation of their field of view.
  2. Current technologies mostly provide spatial visual guidance (indicating where to look) but lack temporal visual guidance (indicating when to look).
  3. Temporal guidance is particularly crucial in videos with fast-paced events or critical event beginnings, as users may miss key moments while adjusting their perspective.

• Significance: The application of 360-degree videos in immersive experiences is becoming increasingly widespread, including in tourism, documentaries, horror genres, and games. The lack of effective spatiotemporal guidance for viewers' perspectives can negatively impact audience experience and the storytelling capabilities of content creators.

• Research Motivation and Related Work:

  1. Existing studies mainly focus on spatial navigation and gaze pattern analysis, with few exploring the impact of temporal guidance on user experience.
  2. The authors aim to address the gaps in existing research and explore a method that combines spatial and temporal visual guidance.

Solution

• Proposed Solution: RadarVR, a novel visual guidance tool that integrates spatial and temporal guidance to provide viewers of 360-degree videos with multidimensional information about Regions of Interest (ROI).

• Innovations:

  1. Innovatively combines radar-based spatial navigation with temporal guidance inspired by rhythm games.
  2. Encodes the direction and timing of ROIs as moving wedges to help users plan head movements.
  3. Merges existing spatial navigation (minimap radar) with visual metaphors from rhythm game design.

• Implementation Steps and Key Technologies:

  1. RadarVR is based on predefined ROI data input, including the spatial coordinates (bounding box) and appearance/disappearance times of the ROI.
  2. ROIs are visualized as wedge-shaped patterns that move radially around the radar to reflect the passage of time; when the wedge reaches the radar center, the ROI becomes immediately visible.
  3. The system also provides directional guidance (e.g., using green arrows to indicate vertical direction when the ROI is outside the user's view).
  4. Developed on the Unity platform and integrated with Meta Quest 2 headsets, using plugins like AVPro Video for video playback and synchronization.

Research Outcomes

• Specific Results:

  1. Increased ROI Visibility: Experiments showed that adding temporal guidance (RadarVR's ST mode) helped users see more ROIs overall, with a 6% increase compared to spatial guidance (RadarVR's S mode).
  2. Significant Improvement in ROI Initial Visibility: Temporal guidance enabled viewers to see the beginning of ROIs more frequently, with an 81% increase.
  3. Longer Viewing Duration: With the ST mode, viewers spent 21% more time observing ROIs.

• Comparative Advantages: Compared to traditional spatial visual guidance methods, RadarVR's temporal feedback is particularly effective in the following scenarios:

  • Short-lived events in fast-paced videos
  • Videos with spatially dispersed content requiring frequent head movements
  • Scenes where the beginning of events is critical

• Experimental or Evaluation Results:

  1. Quantitative Analysis: User tracking data indicated that the ST mode significantly increased ROI initial visibility and sustained hit rates.
  2. Qualitative Feedback: Temporal guidance significantly helped users plan their viewing behavior, though individual preferences varied depending on user goals (exploration vs. precise observation) and psychological needs.
  3. Usability and Stress: NASA-TLX evaluations showed no significant differences in overall cognitive load between the two modes, but the ST mode resulted in lower stress levels when tracking short-term events.

• Limitations and Future Directions:

  1. RadarVR currently requires manual ROI annotation; future work could explore automated or interactive annotation techniques.
  2. The study focused on tasks where users aim to view all important regions in a video; future research could investigate more natural viewing scenarios or other quantitative metrics (e.g., immersion, distraction feedback).
  3. The relative importance of multiple dimensions in temporal guidance (e.g., event arrival time vs. event duration) warrants further exploration.

Conclusion

The RadarVR project investigates the impact of spatiotemporal visual guidance on user experience, demonstrating that temporal feedback can effectively enhance visual guidance. In the field of virtual reality storytelling, RadarVR provides new tools and perspectives for content creators while opening up new challenges and opportunities for exploring user preferences, personalized design, and reducing cognitive load.