CO/DA: Live-Coding Movement-Sound Interactions for Dance Improvisation

Document Title

CO/DA: Live-Coding Movement-Sound Interactions for Dance Improvisation

Document Information

• Subject Area: Human-Computer Interaction (HCI), Dance and Sound Interaction Design
• Keywords: Live coding, dance, improvisation, embodied interaction, movement, design

Research Background and Problem

• Identified Problems or Challenges:

  • The design of movement-sound interactions in dance requires more real-time and dynamic tools. Existing tools are often based on fixed interaction models, making them less adaptable to improvisation and complexity.
  • While live coding has been widely applied in music and visual arts, its integration with bodily movements and sound remains underexplored.

• Significance of the Problem:

  • Exploring real-time interactions where movements drive sound generation contributes to artistic creation and enhances understanding of the relationship between the body and technology.
  • This exploration offers significant academic and practical value in improving dance improvisation and designing more flexible interaction tools.

• Research Motivation and Related Work:

  • Although the dance community has seen various technological interventions, the value of the action-perception-feedback loop in improvisation remains underexplored.
  • Modern interaction design methods and tools (e.g., embodied sketching and somaesthetic design) provide important inspiration for this research.
  • Building on prior experience with sound and movement interaction, the authors extend their interactive design projects to offer new avenues for experimentation and improvisation.

Solution

• Proposed Method or Solution:

  • Developed a live coding environment called "CO/DA" that supports interaction mapping between dance movement data and sound.
  • Through an embodied improvisation approach, sensors capture dance movement data, which is then used to generate interactive sound feedback via live coding.

• Innovations:

  • CO/DA supports real-time stream processing and dynamic code generation, enabling improvisation and complexity suitable for non-linear creative contexts like dance.
  • Emphasizes event-driven architecture and functional signal processing, allowing interaction patterns to transcend predefined constraints and support improvisational exploration.
  • Integrates machine learning techniques to capture movement features and build dynamic models, enriching the relationship between dance movements and sound feedback.

• Implementation Steps and Key Technologies:

  • Movement Sensing: Utilizes sensors like the Myo Armband to capture dancers' movement data and transmit it in real-time to CO/DA.
  • Data Processing and Mapping: Defines mapping relationships between movement data and sound parameters through a flexible API.
  • Sound Synthesis: Employs sample-based sound synthesis techniques (e.g., granular synthesis and audio splicing) to generate rich sound feedback.
  • Real-Time Visualization and Interaction: Provides dynamic data stream tracking with a graphical interface for real-time monitoring and adjustments.
  • Movement Learning and Recognition: Integrates an interactive machine learning module to capture and dynamically construct a vocabulary of movements.

Research Outcomes

• Specific Achievements:

  • Developed a fully functional live coding environment, CO/DA, which has been released as open-source software.
  • Facilitated real-time interaction and improvisation practices based on dance, identifying key design elements such as balancing control and ambiguity, and navigating persistence and abruptness.

• Advantages Compared to Existing Solutions:

  • CO/DA supports dynamic and complex movement-sound interaction patterns, avoiding the design space limitations of traditional tools.
  • This live coding environment emphasizes interaction design thinking that embraces uncertainty and generativity, adapting to the evolving needs of artistic creation.

• Experimental or Evaluation Results:

  • Over two years, the authors documented 23 improvisation sessions, analyzing how CO/DA inspired creativity and interaction.
  • Experiments revealed that dynamic design increased dancers' immersion with the technology and enhanced the playfulness and complexity of movement-sound interactions.

• Limitations and Future Directions:

  • The design and practice of CO/DA are primarily based on personal artistic exploration, making it highly subjective and less directly applicable to other contexts.
  • The complexity of live coding may pose a usability barrier for users without technical backgrounds.
  • Future directions include optimizing the platform's usability, exploring more dance design scenarios, and studying its promotion and application in non-professional environments.

Conclusion and Academic Contributions

• Through the development of CO/DA and two years of improvisational practice, the authors explored the possibilities of technology-body interaction and proposed new principles for creative design.
• This research contributes to the HCI and art design fields by providing practical experiences and academic reflections on using live coding as a tool for movement interaction, including approaches to improvisation, non-linear creation, and embracing uncertainty.