Here and Now: Creating Improvisational Dance Movements with a Mixed Reality Mirror

Title of the Paper

Here and Now: Creating Improvisational Dance Movements with a Mixed Reality Mirror

Paper Information

• Field of Study: Human-Computer Interaction Design and Performing Arts (specifically the intersection of dance and technology)
• Keywords: Dance, Mixed Reality, Augmented Reality, Mirror, Improvisation

Research Background and Problem

• Identified Problems or Challenges:

  1. Dance is inherently complex, particularly in improvisational creation, which relies heavily on dancers' bodily experience and spatial perception.
  2. Current technological tools are inadequate in capturing dance movements and assisting creation, especially in providing real-time feedback and multidimensional spatial awareness.
  3. While traditional studio mirrors are widely used in dance training, they may lead to an overemphasis on external body image, potentially limiting the freedom of improvisation.

• Significance of the Problem:

  1. Dance creation depends on dancers' profound understanding of space, time, and the body. Technological tools can aid in developing this sensitivity.
  2. Mixed reality technology, with its potential for real-time interaction and 3D spatial visualization, can naturally integrate into the dance creation process, avoiding the limitations of immersive head-mounted devices.

• Research Motivation and Related Work:

  1. Inspired by William Forsythe's Improvisation Technologies, which uses computer-generated imagery to visually analyze dance space and movement.
  2. Historical attempts include Étienne-Jules Marey's motion trajectory capture and Eadweard Muybridge's motion imagery, but these did not address the need for real-time creation and immediate feedback.
  3. The widespread use of mirrors in dance studios and the rapid development of mixed reality technology present an opportunity to design natural and effective dance-assistive technologies.

Proposed Solution

• Proposed Solution: Design a mixed reality (MR) mirror with four interactive visualization features to help dancers better perceive their bodies, movement trajectories, and spatial structures during improvisation.

  1. Point-point-Line (PL): Creates straight lines between dancers' body joints to visualize geometric relationships in movement space.
  2. Air Drawing (AD): Draws the trajectory of dancers' hand movements, showcasing the progression of motion.
  3. Delayed Presence (DP): The mirror renders a delayed 3D capture of the dancer's body in real-time, allowing interaction with their past self.
  4. Embodied Estrangement (EE): Replaces the dancer's reflection with a featureless virtual avatar to explore unfamiliar bodily perceptions and creative possibilities.

• Innovative Aspects:

  1. Combines real-time 3D rendering with mirror reflections, surpassing the limitations of traditional video recording and 2D visualization.
  2. Utilizes virtual avatars and motion trajectories to inspire new bodily awareness and movement exploration during improvisation.
  3. Designs interaction technologies tailored to dancers' improvisational habits, avoiding disruptions to the creative flow.

• Implementation Steps and Key Technologies:

  1. The prototype uses Azure Kinect for full-body motion tracking and Unity to develop the four visualization features.
  2. Research methods include technical testing, scenario storyboard discussions, and low-tech prototype design to gather user feedback and creative design suggestions.

Research Outcomes

• Specific Outcomes:

  1. The MR mirror enhanced dancers' sensitivity to space and time, using visual feedback to help them create new movements.
  2. The DP feature was particularly favored by professional dancers for its opportunities for multi-layered interaction with time and the body.
  3. The EE feature encouraged dancers to break free from habitual performance patterns, fostering new modes of expression.

• Advantages Over Existing Solutions:

  1. Compared to traditional mirrors and video recordings, the MR mirror enables real-time interaction with dancers, creating richer creative possibilities.
  2. Reduces the technical barriers associated with head-mounted devices, offering a more natural visualization experience.

• Experiment or Evaluation Results:

  1. In group experiments, dancers improvised in front of the mirror, and their interactions with the mirror were recorded.
  2. Feedback from expert and student dancers included insights into creative inspiration, changes in self-perception, and expectations for future functionalities.
  3. Features such as rendering spatial volumes in reflections, historical motion trajectories, and dynamic feedback were highly praised.

• Limitations and Future Directions:

  1. Some novice dancers showed lower acceptance of technological innovations, preferring traditional training methods.
  2. The limited field of view of the Kinect device restricted full-body motion tracking.
  3. Future research could explore the varying needs of dancers with different levels of experience and improve device compatibility and feature diversity.