Strollytelling: Coupling Animation with Physical Locomotion to Explore Immersive Data Stories

Research Background and Problem

Identified Issues or Challenges

• Narrative visualization techniques are often used to combine data visualization and storytelling to convey information to audiences. However, most such techniques are limited to traditional screen-based environments (e.g., "scrollytelling" techniques in desktop interfaces).
• While virtual roaming techniques in virtual reality (VR) allow users to explore immersive data stories, research suggests that physical roaming has potential advantages in enhancing immersive experiences.
• Current design frameworks lack support for integrating physical movement with the progression of animated data stories.

Significance

• Immersive data storytelling can enhance audience understanding and information retention through emotional and cognitive engagement.
• Incorporating physical movement in immersive environments not only has the potential to increase user engagement but also to improve information recall while mitigating cybersickness.

Research Motivation and Related Work

• The authors aim to extend traditional "scrollytelling" techniques to immersive environments through new technologies.
• The proposed "strollytelling" technique is inspired by traditional web-based scrolling storytelling and the benefits of physical roaming (e.g., higher immersion).
• Related research includes narrative visualization, storytelling techniques in VR, and action-based cognitive theories, which emphasize the role of physical movement in information processing.

Solution

Proposed Method or Solution

• The authors propose the "strollytelling" technique, a novel storytelling method for immersive data narratives that tightly integrates users' physical movement with the progression of story animations.
• The animation playback is triggered by the user's position in physical space, enabling dynamic visualization content to unfold as the user moves.
• Detailed design considerations are provided, such as mapping physical space to virtual space, arranging data story points, and visual layout strategies.

Innovations of the Solution

• By controlling animation frames based on the user's current position, the technique achieves a deep integration of physical space and virtual environments.
• The introduction of pause points allows users to stop at specific locations to observe and interact more closely, enhancing the sense of immersion.
• The technique introduces a new design space and framework in immersive data storytelling, combining movement and visual effects to improve cognitive and experiential quality.

Implementation Steps and Key Techniques

• Define key design elements, such as the "strolly path," data story points, animations, and pause points.
• Develop technical algorithms to calculate the user's distance from the nearest path segment in the virtual environment and adjust animations accordingly.
• Provide an iterative design process, including selecting and adapting data stories, planning physical and virtual layouts, animating data points, and developing the virtual environment.

Research Outcomes

Specific Results

• Design Validation: The authors validated the effectiveness of "strollytelling" through design layouts and user experiments.
• User Experiment: A total of 26 participants completed the experiment, which tested data recall, engagement (immersion), and sense of agency under two conditions (physical roaming and virtual roaming).
• Experimental results showed that most participants preferred "strollytelling" and reported higher immersion and experiential quality.

Advantages Compared to Existing Solutions

• Compared to traditional virtual roaming techniques, "strollytelling" enhances user immersion through physical roaming.
• Participants experienced significantly improved interactivity and control over the data story, aligning with the goals of interactive storytelling.
• Cybersickness was noticeably reduced, and the experience felt more natural.

Experimental or Evaluation Results

• Quantitative Results:
  • In terms of immersion, "strollytelling" scored significantly higher than virtual roaming (P-value = 0.016).
  • Among participants, 17 out of 26 preferred "strollytelling."
  • Recall tests and sense of agency showed no significant differences.
  • Cybersickness scores showed no statistically significant differences between the two conditions.
• Qualitative Feedback:
  • Participants noted that the mapping between physical movement and animation greatly enhanced the experience.
  • The virtual environment and sound effects helped to amplify the storytelling immersion.

Limitations and Future Directions

• The physical constraints of the experimental space may have limited user experiences and the generalizability of the results.
• The distance and number of data story points were restricted by the size of the physical space.
• The experimental group primarily consisted of university students, which may not represent a broader demographic.
• Future work:
  • Explore the use of treadmills or optimized technologies to address space limitations.
  • Investigate the impact of different data story structures (e.g., branching paths).
  • Extend experiments to a broader and more diverse participant group, including non-mobile populations.
  • Develop "adaptive strollytelling" to dynamically adjust story content based on physical space.

Overall, the proposed "strollytelling" technique represents a groundbreaking contribution to the field of immersive data storytelling, pioneering a new dimension that integrates physical movement with narrative data presentation. This innovation holds significant potential for applications in education, museum design, journalism, and beyond.