Can You Hear My Heartbeat?: Hearing an Expressive Biosignal Elicits Empathy

Title of the Paper

Can You Hear My Heartbeat?: Hearing an Expressive Biosignal Elicits Empathy

Paper Information

• Research Area: Human-Computer Interaction (HCI), Affective Computing, Physiological Signal Perception
• Keywords: Empathy, Emotion Recognition, Sound, Music, Heart Rate, Emotional Communication, AAC (Augmentative and Alternative Communication), ASD (Autism Spectrum Disorder)

Research Background and Problem

• What problems or challenges did the authors identify?

  • Empathy is a key capability in human social interactions, and its enhancement can foster social cooperation and compassion. However, existing research has primarily focused on visual representations of physiological signals, with the effects of auditory representations of physiological signals (e.g., heartbeat) remaining underexplored.
  • Designing and evaluating interactive systems that effectively elicit empathy between individuals remains a significant research challenge.
  • Most current studies share heart rate information through visualization, which may not be suitable in certain contexts, such as for visually impaired individuals or scenarios requiring non-visual solutions.

• Why is this issue important?

  • Technologies that enhance empathy can improve interpersonal relationships and drive positive societal change, especially in supporting marginalized groups such as individuals with autism.
  • Exploring the role of auditory physiological signals (e.g., auditory heartbeat) can expand the boundaries of affective computing and has practical applications in fields such as music therapy.

• Motivation and Related Work

  • Conveying physiological signals (e.g., heart rate) can reflect an individual's internal emotional state and enhance interpersonal intimacy and emotional connectedness.
  • Previous studies have shown that sharing heartbeat signals has the potential to support the development of empathy, and that music and acoustics can enhance the perception of others' emotional states through emotional contagion mechanisms.
  • The motivation of this study is to verify whether auditory heartbeats can elicit cognitive and emotional empathy responses in users through emotion recognition tasks.

Solution

• What methods or solutions did the authors propose?

  • The authors proposed using auditory heartbeats as an empathy intervention, combined with visual facial cues, to design experiments that systematically evaluate the impact of auditory heartbeats on emotion recognition and emotional resonance.
  • They examined the effects of heartbeat signals, particularly whether heartbeat signals with fast or slow rhythms that align with visual facial expressions can elicit stronger empathy.

• What are the innovative aspects of the solution?

  • This is the first study to introduce auditory heartbeats into visually guided emotion recognition tasks and experimentally validate the impact of auditory heartbeats on multiple dimensions of empathy.
  • The study innovatively applies experimental designs to distinguish between congruent and incongruent auditory-visual contexts and observes the regulatory effects of heartbeat rhythms on listeners' emotional responses.

• What are the implementation steps? What key technologies were used?

  • Experiment Details:
    1. A multi-condition experiment was designed, including four conditions: visual-only, auditory-only (slow/fast heartbeat), and combined auditory-visual (matched/mismatched).
    2. The "Reading the Mind in the Eyes Test" (RMET) was used as a visual emotion recognition tool, with experiments divided into baseline and intervention groups.
  • Technologies and Experimental Details:
    • Physiological Signal Generation: Auditory heartbeat signals with controllable rhythms were generated using computer music software (e.g., SuperCollider).
    • Data Collection: Participants' changes in emotion recognition (cognitive empathy) and self-reported emotional resonance (emotional empathy) were recorded.
    • Data Analysis: Logistic regression was used to quantify the impact of conditions on changes in emotion recognition, and generalized linear mixed models (GLMM) were employed to analyze differences in emotional resonance scores.

Research Outcomes

• What specific results were achieved?

  • Auditory heartbeats enhanced participants' emotional cognitive abilities (increased the probability of changes in emotion recognition) and improved emotional resonance scores.
  • Adding auditory heartbeats to visual facial tasks enhanced cognitive-level emotional identification. Emotional resonance scores were significantly higher when the auditory heartbeat rhythm matched the visual emotion.

• What advantages does it have compared to existing solutions?

  • Auditory heartbeats overcome the limitations of traditional visually dominant presentations, providing a novel and effective solution for scenarios where visual aids are not feasible.
  • The study demonstrated the potential of auditory interventions to enhance empathy, particularly through the combination of musical elements (rhythm) and physiological signals.

• What were the experimental or evaluation results?

  • A 21% increase in the probability of participants changing their emotion recognition judgments was observed. Emotional resonance scores were significantly higher when auditory heartbeats aligned with visual facial expressions.
  • The system validated the positive effects of auditory heartbeats as an empathy intervention tool across multiple dimensions and confirmed the role of heartbeat signals in emotional communication.

• Limitations and Future Directions

  • Limitations:
    • The selection of heartbeat rhythms was relatively extreme (only slow and fast); future work could include neutral rhythms or finer-grained rhythm classifications.
    • The static visual stimuli used in the RMET test limited the complexity of real-world emotional expressions.
    • The experiments were primarily conducted in controlled laboratory environments, and further validation of real-world usability and ecological validity is needed.
  • Future Directions:
    • Explore the integration of more dynamic emotional cues (e.g., dynamic facial expressions) and further enhance ecological validity.
    • Conduct comprehensive evaluations of the system's potential for specific groups (e.g., individuals with autism).
    • Integrate auditory heartbeat signals into real-time social interaction systems to support emotional communication for both able-bodied and vulnerable populations.