DanXeReflect: Interacting with the Spatio-Temporal Past Movements for Embodied, Reflective Choreographic Collaboration

Paper Title

DanXeReflect: Interacting with the Spatio-Temporal Past Movements for Embodied, Reflective Choreographic Collaboration

Publication Info

• Topic area: XR-based tools for reflective choreographic collaboration
• Keywords: XR, choreography, motion capture, embodied interaction, reflective collaboration, asynchronous feedback, 3D avatars, annotation tools, dance rehearsal, spatio-temporal data

Background and Problem

• Problem / challenge: Traditional video-based tools for choreographic reflection are limited by their flat, screen-bounded format, which detaches feedback from the body and spatial context. These tools fail to capture the embodied, dynamic, and spatial nature of choreographic feedback.
• Significance: Addressing these limitations can enhance choreographic collaboration, enabling dancers and choreographers to reflect and exchange feedback asynchronously while preserving the embodied nature of their practice.
• Motivation and related work: Prior systems in XR and video annotation have focused on training, individual practice, or live performance but lack support for asynchronous, distributed choreographic collaboration. Existing tools like Choreographer’s Notebook and iDanceForms are constrained by linear navigation and abstract annotations, while XR systems like WhoLoDance and YouMove emphasize real-time feedback rather than reflective dialogue.

Solution

• Proposed approach: DanXeReflect, an XR system that transforms rehearsal videos into interactive 3D avatars, enabling embodied reflection and asynchronous collaboration through pose-based searching, body-anchored annotations, and revision-by-demonstration.
• Novelty:
  1. Repositions spatio-temporal movement data as a collaborative communication anchor.
  2. Introduces embodied interaction techniques, including pose-based searching, body-anchored annotation, and revision-through-demonstration.
  3. Provides empirical insights into how embodied motion representations enable distributed, asynchronous collaboration.
• Procedure and key techniques:
  1. Video-to-3D Avatar Reconstruction: Converts rehearsal videos into spatio-temporal 3D avatars using a motion processing pipeline.
  2. Embodied Reflection Techniques: Enables pose-based searching, real-time embodied revisions, and joint-specific annotations within a virtual XR studio.
  3. Annotation Storage: Maps annotations to temporal segments and specific body parts, preserving their spatial and temporal context.
  4. Review and Re-engagement: Allows users to revisit and refine annotations and revisions asynchronously.

Results

• Concrete findings:
  • Embodied query allowed users to locate choreography segments through kinesthetic recall, facilitating reflection on overall sequencing and flow.
  • Embodied revisions enabled side-by-side comparisons of original and alternative movements, enhancing clarity and precision in feedback.
  • Body-anchored annotations improved communication clarity by linking feedback directly to specific joints and movements.
  • Participants articulated movement qualities (effort, space, shape, phrasing) more effectively using multimodal feedback in the 3D environment.
• Advantage over baselines:
  • Compared to flat video, DanXeReflect provided clearer structural insights into choreography, supported asynchronous collaboration, and enabled multimodal, body-specific feedback.
  • Unlike prior XR systems, it emphasized reflective dialogue and asynchronous use rather than real-time training or performance augmentation.
• Experiments / evaluation:
  • Conducted a user study with nine participants (aged 19–28, diverse dance backgrounds) who used DanXeReflect for self-reflection and peer feedback tasks.
  • Tasks included reviewing feedback, annotating their own performances, and providing feedback on others’ performances.
  • Analysis focused on usability, movement articulation, and collaborative potential, using Laban Movement Analysis (LMA) as a framework.
• Limitations and future work:
  • Limited avatar fidelity for certain genres (e.g., hip-hop) and expressive cues (e.g., flow, facial affect).
  • Static segmentation misaligned with phrasing; future systems should allow dynamic, user-adjustable segmentation.
  • Only upper-body tracking was supported; full-body tracking could enhance revision capabilities.
  • Lab-based evaluation limits understanding of real-world rehearsal integration; long-term, in-the-wild deployment is needed.

Summary

DanXeReflect transforms rehearsal videos into interactive 3D avatars, enabling embodied, asynchronous choreographic collaboration. By supporting pose-based searching, body-anchored annotations, and embodied revisions, the system enhances clarity, precision, and self-reflection. A user study demonstrated its ability to deepen choreographic dialogue and expand access to reflective practices beyond the studio. While limitations in avatar fidelity, segmentation, and tracking remain, DanXeReflect highlights the potential of XR to reframe video as an embodied resource for distributed collaboration.