ChoreoCraft: In-situ Crafting of Choreography in Virtual Reality through Creativity Support Tool

Research Background and Issues

• Issues and Challenges: With the increasing demand for innovation and rapid choreography creation driven by digital entertainment and social media, choreographers face unprecedented pressure. Traditional choreography processes rely heavily on memory, encounter bottlenecks during iterative creation, and lack sufficient abstract feedback, which limits the creative potential of choreographers.
• Significance: Choreography is a critical component of dance art, requiring creativity and precise movement design. However, traditional work environments (such as dance studios and mirrors) are spatially and practically constrained. Additionally, the lack of data-supported feedback may hinder choreographers from achieving perfection and continuity in their movements.
• Research Motivation and Related Work: Choreography assistance tools have seen some development, but most focus on high-level conceptual planning and simple dance movement generation, offering limited in-depth feedback or real-time support. Current research centers on applying VR technology to the practical choreography process, addressing creative barriers and enhancing efficiency.

Solution

• Proposed Approach: ChoreoCraft is a VR-based choreography assistance tool that integrates real-time motion capture, a dance movement suggestion system, and systematic movement analysis functionalities to support creative expression and improve efficiency in choreography.

1. Movement Suggestion System:
   • Provides suggestions based on music and previously created movements, using an innovative algorithm, DanceDTW, to calculate the similarity of dance movements and ensure harmony between suggested movements, music, and prior choreography.
   • Utilizes the Spotify API to extract data from online music and videos, employing the WHAM algorithm to convert videos into 3D dance poses.
2. Movement Analysis System:
   • Offers quantitative feedback, such as balance, stability, and joint activity analysis, aiding choreographers in optimizing movements.
   • Visualization tools highlight key joints and spatial distribution, allowing choreographers to intuitively understand movement flaws.
3. Other Innovative Features:
   • Snapshot functionality for recording choreography segments, supporting playback and seamless movement extension.
   • Immersive creation space built with avatars in the VR environment, reducing memory reliance.

Research Outcomes

Specific Results

1. Experimental Results:

   • The dance movement suggestion system demonstrated significant advantages in stimulating creativity and improving work efficiency.
   • The movement analysis system achieved notable progress in improving balance and stability, with participants showing an average improvement of 7.92% (balance score).
   • The Snapshot functionality effectively reduced memory dependency, with user satisfaction reaching 5.95/7 points (Likert scale).

2. Comparison with Existing Solutions:

   • Compared to traditional mirror-based choreography methods, ChoreoCraft overcomes spatial and temporal constraints, significantly enhancing creative efficiency.
   • The DanceDTW algorithm's accuracy far surpasses other benchmark methods (such as Bailando and EDGE algorithms), achieving an accuracy rate of 89.66%.

3. User Feedback:

   • Users generally found the system's suggested dance movements creative and inspiring for new ideas.
   • The movement analysis system's score guidance helped dancers improve the precision of their movements through data-driven feedback.

Limitations and Future Directions

1. Technical Limitations:

   • The weight of VR devices affects performance during high-intensity dance movements.
   • The Azure Kinect-based motion capture system cannot accurately capture subtle movements (such as facial expressions and finger gestures).

2. Future Improvements:

   • Enhance the lightweight design of VR devices to reduce physical burden on users.
   • Incorporate wearable sensor systems to further improve motion capture accuracy.
   • Design automated movement improvement guidance for beginners, making the system more intelligent.

3. User Experience Optimization:

   • Provide more detailed training and instructional tutorials to help choreographers quickly familiarize themselves with various metrics of the movement analysis functionality.
   • Add user customization options, such as selecting specific analysis factors, to support personalized choreography processes.

In conclusion, ChoreoCraft not only improves the choreography process but also demonstrates the feasibility and potential of VR-based creativity support tools in the field of artistic expression. It is poised to become an essential tool for the next generation of choreographers.