Behind the Scenes: Adapting Cinematography and Editing Concepts to Navigation in Virtual Reality

Title of the Paper

Behind the Scenes: Adapting Cinematography and Editing Concepts to Navigation in Virtual Reality

Paper Information

• Subject Area: Virtual Reality (VR) Navigation and User Experience Optimization
• Keywords: Virtual Reality, Navigation, Cinematography, Multi-objective Optimization, User Engagement

Research Background and Issues

• Identified Problems or Challenges:

  1. In virtual reality, the "teleportation" navigation method alleviates VR motion sickness (e.g., nausea, loss of sense of direction), but it reduces users' spatial awareness, leading to missed important information in the environment.
  2. Existing navigation improvement studies primarily focus on task completion efficiency (e.g., accuracy in target selection) and rarely explore how to enhance the non-instrumental (non-task-oriented) dimensions of user experience.

• Significance:

  • Reduced spatial awareness can lead to decreased task completion efficiency; higher user engagement and aesthetic appeal of the environment can significantly improve VR user experience.
  • Optimizing VR navigation experience without compromising user control or inducing VR motion sickness is a crucial research direction for expanding the scope of VR applications.

• Research Motivation and Related Work:

  • Existing studies have made progress in improving smooth locomotion and discrete movement (e.g., teleportation), but few have explored the potential advantages of introducing "continuity rules" from film editing into VR navigation.
  • This study leverages film editing rules (e.g., the 30-degree rule, rule of thirds) to enhance the aesthetic and user experience of VR teleportation navigation.

Solution

• Method or Solution:
  A novel teleportation navigation method named ACTIVE (Adaptive Cinematic Teleportation In Virtual Environments) is proposed, which incorporates film editing rules into VR teleportation to dynamically adjust the user's camera position and orientation, thereby enhancing the aesthetic appeal and user experience of the scene.

• Innovations:

  • Redefining Teleportation: Treating teleportation as a "cut" in filmmaking and utilizing continuity editing rules to make transitions between teleports smoother and visually appealing.
  • Optimization Method: Translating film editing rules into optimization objectives using the Von Mises distribution to calculate the optimal camera adjustments in real-time.
  • Multi-objective Optimization: Simultaneously considering multiple rules (e.g., rule of thirds, forward direction) to determine the optimal camera angle and position.

• Implementation Steps and Key Techniques:

  1. Teleportation Trigger: Users select a target location and trigger teleportation.
  2. Camera Relocation: Adjust the camera position using the 30-degree rule and "headroom" considerations.
  3. Camera Reorientation: Calculate the optimal angle and adjust the camera direction based on editing rules (e.g., rule of thirds, 180-degree rule).

Research Outcomes

• Specific Results:

  1. Proposed the ACTIVE model, which integrates film editing rules into VR navigation.
  2. Experimental validation showed that ACTIVE significantly enhances the aesthetic appeal and user engagement in VR, with an overall user experience score improvement of 8.6%.
  3. ACTIVE showed no significant difference compared to traditional teleportation methods in terms of users' memory accuracy of environmental entities, VR motion sickness, and presence.

• Advantages (Compared to Existing Solutions):

  • Improved visual appeal and consistency of the scene after teleportation without compromising user control or inducing additional motion sickness.
  • Addressed the shortcomings of traditional teleportation methods where users needed extensive manual adjustment of their viewpoint.

• Experiment and Evaluation Results:

  • User Engagement: On the "User Engagement Scale" (UES), ACTIVE achieved a 17.6% improvement in the "aesthetic appeal" dimension.
  • User Memory: Although users' recall of virtual scene entities did not significantly improve, there was no negative impact on memory.
  • Motion Sickness: ACTIVE showed no statistically significant difference compared to traditional teleportation methods in the "Simulator Sickness Questionnaire" (SSQ) results.
  • Dimensional Evaluation: Users expressed high control satisfaction with ACTIVE through the "Presence Questionnaire," with no significant negative experiences.

• Limitations and Future Directions:

  • Limitations:
    1. Relies on manual annotation of virtual environment entities and path vectors, increasing workload.
    2. Experiments were conducted in a single virtual environment setting, which may not cover the applicability of ACTIVE in complex or non-linear scenarios.
    3. Lacks real-time tracking data of user gaze/attention.
  • Future Directions:
    1. Automate the annotation of environment entities and path vectors to enhance scalability.
    2. Extend validation across different types of environments (e.g., games, cultural heritage exhibitions, immersive films) to test ACTIVE's generalizability.
    3. Integrate eye-tracking technology for deeper analysis of changes in user attention.

Conclusion

ACTIVE provides an innovative VR teleportation navigation method by introducing cinematography and editing rules to significantly enhance the aesthetic appeal of VR scenes without compromising user control or increasing the risk of VR motion sickness. This study lays the foundation for creating navigation experiences in virtual reality that foster higher user engagement.