RetroSketch: A Retrospective Method for Measuring Emotions and Presence in Virtual Reality

Research Background and Issues

• Identified Problems or Challenges: A core challenge in virtual reality (VR) experience design is accurately and continuously measuring users' emotions and sense of presence. Existing methods, such as the Experience Sampling Method (ESM), interrupt the experience and lack data continuity. Physiological sensing methods, while offering continuous measurement, are difficult to analyze effectively and are susceptible to noise.
• Significance: Immersion and emotional experience in VR are critical factors influencing user impact, directly affecting training outcomes, gaming engagement, and even therapeutic interventions. Measuring emotions and presence can help developers optimize designs and improve the quality of VR experiences.
• Research Motivation and Related Work: Previous studies have widely adopted ESM and physiological signals as primary methods for measuring emotions, but these approaches have significant limitations. This research aims to propose a new solution to supplement or replace existing methods, addressing these limitations and improving the quality of emotion quantification.

Solution

• Proposed Method: The authors introduce a novel measurement method called RetroSketch. This retrospective tool for measuring emotions and presence allows users to watch video replays of their VR experiences, sketch emotion fluctuation graphs, and annotate key events.
• Innovations:
  • RetroSketch avoids interrupting the experience while providing highly continuous emotion and presence measurement data.
  • By combining video replay, it assists users in retrospective emotion evaluation and allows annotation of key events and descriptions of emotional experiences, expanding qualitative information.
  • RetroSketch is available in both digital and paper-based formats, offering ease of use and flexibility.
  • It provides data export functionality for subsequent analysis.
• Implementation Steps and Key Techniques:
  1. Record the user's VR gameplay experience and replay the first-person perspective video.
  2. Users draw emotion and presence rating curves and annotate key emotional events.
  3. In the digital version of RetroSketch, an interactive timeline cursor synchronizes the video with the user-drawn curve, supporting detailed annotations.
  4. Collect and save users' quantitative data, key annotations, and descriptive information.

Research Outcomes

• Specific Results and Advantages:
  • RetroSketch shows a high correlation with ESM in emotion measurement (strong correlation), validating its effectiveness.
  • RetroSketch significantly outperforms ESM in terms of emotion measurement coverage and data granularity, capturing more dynamic emotional changes.
  • RetroSketch avoids common biases in ESM, such as social desirability bias and observer effects.
  • RetroSketch performs better than ESM in handling rapidly changing physiological signals (e.g., skin conductance response) and is comparably effective with slower-changing signals (e.g., heart rate variability).
  • The annotations and key event information it collects are consistent with quantitative data and can be processed using automated emotion analysis models.
• Experiments and Evaluation Results:
  • The study conducted user experiments with 140 participants across five different types of VR games, covering racing simulations, open-world exploration, action-horror, and other experiences.
  • The experiments demonstrated that RetroSketch maintains consistent performance across diverse and complex VR scenarios, proving its applicability to different demographic and personality types.
  • RetroSketch has minimal but unpredictable effects on user experience. For instance, ESM increased stress in some games while reducing it in others.
• Limitations and Future Directions:
  • RetroSketch has been tested for VR experiences up to 30 minutes in duration, and its applicability to longer or shorter experiences remains to be explored.
  • Enhancing the reliability of retrospection, especially when the time elapsed since the experience increases, requires further investigation.
  • The performance and experience of RetroSketch in multiplayer scenarios could be a future research direction.
  • The authors suggest exploring the integration of VR replay environments to enhance users' memory recall capabilities, rather than relying solely on video playback.

The above analysis summarizes the research background, innovative solution, experimental results, and future potential of RetroSketch, demonstrating its significant contribution to improving emotion measurement methods and providing new tools and directions for research and applications in the field.