Corsetto: A Kinesthetic Garment for Designing, Composing for, and Experiencing an Intersubjective Haptic Voice

Title of the Paper

Corsetto: A Kinesthetic Garment for Designing, Composing for, and Experiencing an Intersubjective Haptic Voice

Paper Information

• Research Area: Human-Computer Interaction (HCI), Wearable Technology, Embodied Perception
• Keywords: Robotic Textiles, Shape-Changing Interfaces, Haptics, Machine Learning, Voice, Kinesthetic Interaction Design, Microphenomenology

Research Background and Problem Statement

• What problems or challenges did the authors identify?

  • Current haptic interaction design primarily focuses on replicating existing interpersonal tactile interactions, such as handshakes and hugs, while lacking exploration of novel experiences and embodied knowledge.
  • These designs often fail to fully consider the complexities of internal and external bodily perceptions, as well as interactions with other bodies.
  • Existing designs rarely adopt a phenomenological perspective to uncover the tactile experiences introduced by new technologies.

• Why is this problem important?

  • Singing is a complex embodied process that combines internal muscular activity with external expression. Understanding this experience in depth is crucial for exploring intersubjective perceptual interactions.
  • Investigating new methods to enhance interbodily perception and connection through technology holds significant potential for applications in art, education, and interaction design.

• Motivation and Related Work

  • The research is inspired by French phenomenologist Maurice Merleau-Ponty's concept of "intercorporeality," aiming to design interactive technologies that enable novel interbodily empathy experiences.
  • It explores how to design haptic experience technologies that integrate internal and external bodily perception boundaries, particularly in the context of musical performance.

Solution

• What methods or solutions did the authors propose?

  • Development of a haptic garment called "Corsetto," employing pneumatically actuated textile muscle modules to reproduce and extend the tactile gestures of bodily movements during singing.
  • Design of a system architecture for haptic gesture composition and real-time control, incorporating both hardware and software stacks.
  • Integration of machine learning techniques to automatically recognize musical gestures and generate real-time dynamic haptic responses.

• What are the innovative aspects of the solution?

  • Corsetto not only mimics the singer's bodily movements but also adds a "fourth voice" through haptic gestures, bridging external human perception with internal sensations.
  • The system architecture introduces a novel method for composing haptic gestures based on musical scores, pioneering the integration of art, design, and technology in the HCI domain.
  • Pneumatic textile technology provides soft and nuanced responses, significantly surpassing traditional mechanical or vibratory feedback devices.

• What are the implementation steps and key technologies used?

  • Design Phase: Extracting bodily movement characteristics through singing training, kinesthetic understanding, and deconstruction exercises, followed by the creation of wearable pneumatic actuators.
  • Composition Phase: Constructing haptic gesture scores using MusicXML and developing a conversion tool to automatically generate tactile motion instructions from sound gestures.
  • Performance Phase: Testing Corsetto in two immersive opera performances and documenting user experiences through microphenomenological interviews.
  • Key technologies include: pneumatic soft robotic fibers (OmniFiber), machine learning (CNN-based models), and cross-module programming frameworks (FlowIO).

Research Outcomes

• What specific results were achieved?

  • Corsetto enabled synchronization of haptics and audio during musical performances, allowing audiences to tangibly experience the tactile expressions of the singer's bodily movements.
  • The device facilitated audience experiences of blurred boundaries between internal and external perception, fostering new aesthetic and narrative experiences.

• How does it compare to existing solutions?

  • It offers a novel haptic interaction experience that transcends simple imitation, combining subtlety and dynamism.
  • Enables more complex interbodily interactions—from passive sharing of the singer's experience to tactile dialogues with artificial entities, even achieving perceptual boundary fusion.

• What were the experimental or evaluation results?

  • Microphenomenological interviews revealed that Corsetto:
    • Supports the transformation of vocal sensory experiences into tangible bodily haptics.
    • Provides tactile dialogues between the audience and the device, instantly reproducing the singer's bodily perceptions.
    • In certain intense experiences, blurred the boundaries between subject and object for the audience, achieving integration of sound and bodily experience.
  • During performances, audiences perceived the holistic nature of wearable performances, with haptics and sound intertwined, and their perception evolved over time and space.

• Limitations and Future Directions

  • Further testing with a broader audience, especially non-professional music users, is needed.
  • Exploring the extension of pneumatic textile technology to other types of interbodily experiences, such as sports training or everyday interactions.
  • Conducting in-depth analysis of the system framework and user experience, incorporating Corsetto into more performances or diverse application scenarios.