Designing and Evaluating an Advanced Dance Video Comprehension Tool with In-situ Move Identification Capabilities

Title of the Paper

Designing and Evaluating an Advanced Dance Video Comprehension Tool with In-situ Move Identification Capabilities

Paper Information

• Subject Area: Human-Computer Interaction and Dance Learning Technologies
• Keywords: Dance, Learning Dance, Choreography, Human Motion Analysis, Search System

Research Background and Problem

• Identified Issues or Challenges: Dance learners face difficulties in analyzing complex dance videos and taking notes. Existing video search and note-taking tools often require users to know the names or other attributes of dance moves, making the learning process cumbersome. Additionally, users need to leave the video context to search for relevant information, disrupting the learning experience.
• Significance: Dance is a highly visual art form, and learning dance through videos has become an important method. Leveraging machine learning technologies (e.g., human motion recognition) can significantly enhance the learning efficiency of dance learners. However, tools specifically designed for dance move recognition and note-taking assistance remain insufficient.
• Research Motivation: This study aims to develop and evaluate an advanced video comprehension tool integrated with automatic dance move recognition to help learners more effectively take notes and understand video content.

Solution

• Proposed Solution: Develop and evaluate a prototype video comprehension tool with the capability to automatically recognize dance moves based on video segments. The prototype employs the Wizard-of-Oz method to simulate future computer vision technologies.
• Innovations:
  • Integrated automatic dance move recognition functionality, which directly returns a list of dance moves and related metadata based on user-selected video segments.
  • Improved user interface design, such as allowing users to select segments on a timeline and attach notes, while providing multi-angle video displays and move descriptions.
• Implementation Steps:
  1. Conduct interview studies to understand the challenges dance learners face in video analysis and note-taking.
  2. Design a prototype based on interview results, incorporating timeline selection, note-taking features, and an automatic dance move recognition button.
  3. Use the Wizard-of-Oz method to pre-label dance moves to simulate automatic recognition functionality, including timestamps and metadata for the moves.
  4. Evaluate the prototype's effectiveness through experimental studies, comparing it with baseline conditions.

Research Outcomes

• Specific Results:
  • The prototype tool significantly improved the quality of notes recorded by participants, with average scores increasing from a baseline of 5.65 to 8.71 (out of 10).
  • Users reported reduced subjective workload, with NASA-TLX questionnaire results showing significant improvements in "Frustration" and "Performance" metrics.
• Comparison with Existing Solutions and Advantages:
  • More efficient than traditional feature-based search systems, avoiding the need for learners to leave the video context.
  • Provides real-time automatic recognition functionality and integrated information, simplifying retrieval difficulties.
• Experimental or Evaluation Results:
  • In experiments, participants effectively utilized the tool's video segment selection and automatic recognition features, revealing different usage patterns.
  • 80% of participants successfully used the automatic recognition feature during video annotation tasks.
• Limitations and Future Directions:
  • Currently evaluated only on Cuban Salsa dance; other dance types may require further adaptation.
  • The tool's impact on long-term dance learning outcomes, such as improvement in actual performance skills, has not been assessed.
  • Some user feedback indicated overly lengthy result lists and challenges in distinguishing similar moves.
  • Social and feedback functionalities are not yet integrated; future designs could consider supporting social interaction among learners.

Design Recommendations and Extended Applications

• Design Recommendations:
  • Integrate segment selection and annotation functionality on the timeline to provide users with more precise dance move segmentation tools.
  • Add standardized descriptions of moves to help learners master technical terminology.
  • Offer multi-angle video displays and detailed explanations for search results.
• Extended Applications:
  • Design similar tools for other action-based performance scenarios, such as sports movements, medical procedures, and language learning video annotation systems.

Conclusion

This study developed an advanced dance video comprehension tool and demonstrated its effectiveness in improving note quality and reducing workload. Through user-generated automatic recognition tasks from video segments, the tool not only provides new support methods for dance learning and analysis but also introduces new tasks and application directions for human-computer interaction design and computer vision research.