Thinking with Sound: Exploring the Experience of Listening to an Ultrasonic Art Installation

Title of the Paper

Thinking with Sound: Exploring the Experience of Listening to an Ultrasonic Art Installation

Paper Information

• Research Area: Human-Computer Interaction (HCI), Sound Studies, Interaction Design, Ultrasonic Art
• Keywords: Sound installation art, ultrasound, sound interaction, listening experience, sound studies, micro-phenomenology, entanglement theory, diffraction analysis, perception studies

Research Background and Issues

• Identified Problems or Challenges:

  • Current HCI design practices exhibit a visual bias, neglecting the exploration of human-technology interaction through other senses, such as sound.
  • The materiality, temporality, and spatiality of sound are often overlooked by traditional design methodologies.
  • The complex and dynamic nature of interaction experiences necessitates new theoretical frameworks and methodological support.

• Significance:

  • Sound, through its materiality and dynamism, can reveal novel interaction properties, particularly the non-separable relational characteristics (entanglement) between humans and technology.
  • Research on sound offers new directions for reshaping perception frameworks within HCI.

• Research Motivation and Related Work:

  • Sound, as a non-representational medium of communication, not only transforms user experiences but also sparks discussions on post-human perspectives in design.
  • Existing studies have yet to comprehensively explore the experience of sound installation art, particularly how participants perceive, interpret, and interact with sound.
  • Building on relevant sound studies theories (e.g., Turvey, Ballard, Cox), the authors propose an integrated research approach to sound systems.

Proposed Solution

• Methods or Solutions:

  • Based on entanglement theory and the materiality of sound, the authors introduce a novel ultrasonic art installation, Being With The Waves.
  • Employing micro-phenomenological interview methods to deeply explore the complex interaction between human subjects and sound during the listening process.

• Innovations:

  • Designing and studying a "non-symbolic representation" interaction experience, where ultrasonic installation technology interacts with participants through sound itself rather than symbolizing behaviors via sensors.
  • Utilizing diffraction analysis to interpret participant experience data within the framework of sound studies and interaction design theories.
  • Shifting the focus of experience exploration from "user-centered" to a "multi-subject-relational" perspective, introducing the concept of "entangled subjectivity."

• Implementation Steps and Key Technologies:

  1. Design and Technical Implementation:
     • The ultrasonic art installation system consists of "transmitter" and "receiver" technologies.
     • Using amplitude modulation to shift sound signals into the ultrasonic range, enabling listeners to perceive sound through specialized headphones.
     • Designing concealed sound source transmitters, encouraging listeners to explore space and their own positioning to experience the materiality of sound.
  2. Thematic Guidance and Data Collection:
     • Participant Study: Recruiting frequent visitors to art exhibitions to participate in the listening experiment.
     • Listening and Interview Process: Setting up exhibition spaces where participants experience the installation via headphones and engage in semi-structured and micro-phenomenological interviews.
  3. Analysis Methods:
     • Conducting diffraction analysis on micro-phenomenological data, combining sound theory to uncover relationality, dynamism, and bodily intentionality implicit in phenomenological descriptions.

Research Outcomes

• Specific Findings:

  • The sound interaction experience was categorized into four major phenomenon groups: emotional connection, relational shifts, body-sound alignment, and sensor expectations.
  • Listeners established dynamic relationships with sound by controlling body posture, movement, and spatial exploration.
  • The listening experience transcended symbolic interaction, showcasing complex sensory experiences such as stickiness, resistance, and fluidity formed through body-sound interaction.

• Advantages:

  • Compared to traditional methods of "user suggestions or symbolic feedback," the sound materiality-based approach offers greater subtlety, emphasizing dynamic interaction and non-human subjectivity.
  • Diffraction analysis revealed the diversity of participants' sensory experiences, proposing a new interaction design concept prioritizing dynamism and multi-sensory perception.

• Experimental or Evaluation Results:

  • Participants reported multiple significant experiential moments, including interactions with the materiality of sound, bodily sensory adjustments, and various associations.
  • Micro-phenomenological interviews uncovered elusive auditory experience details, highlighting the complexity of sound-body interaction.

• Limitations and Future Directions:

  • Limitations:
    • The participant pool primarily consisted of individuals with artistic or related backgrounds, leaving the generalizability for ordinary audiences to be further validated.
    • Micro-phenomenological methods still face challenges in capturing the dynamics and fluidity of sound.
  • Future Directions:
    • Developing research methods for diverse senses (e.g., tactile, kinesthetic perception).
    • Expanding the application scenarios of sound installations (e.g., public spaces, non-visual-oriented interaction design).
    • Exploring how dynamic sound experiences influence the design process and its philosophical implications.