EmoBalloon - Conveying Emotional Arousal in Text Chats with Speech Balloons

Title of the Paper

EmoBalloon - Conveying Emotional Arousal in Text Chats with Speech Balloons

Paper Information

• Research Area: Human-Computer Interaction (HCI)
• Keywords: speech balloon, text chat, voice input, emotion, computer-mediated communication, paralinguistic cues, emotional arousal, emoticon

Research Background and Problem Statement

• Problems and Challenges:

  • Modern Instant Messaging (IM) and text chat applications are widely used, but these communication methods lack non-verbal cues (e.g., tone, facial expressions, and gestures), which can result in inaccurate emotional expression between senders and receivers.
  • Message recipients often struggle to understand the sender's emotional intent, potentially leading to misunderstandings or communication barriers, such as misinterpreting negative or neutral emotions.
  • While existing solutions (e.g., emoticons, font styles, or animated text) have made some improvements, they remain limited—they often fail to reflect the sender's genuine emotional state.

• Significance of the Problem:

  • Emotional communication is crucial for building strong social connections and personal well-being. Improving emotional transmission in the constrained medium of "text chat" is a key challenge, particularly in reducing the emotional perception gap between senders and receivers.

• Motivation and Related Work:

  • Previous research has explored enhancing emotional expression through emoticons and font styles, but these approaches often fail to convey emotional intensity (e.g., levels of emotional arousal).
  • In Japanese manga, "speech balloons" have been studied in relation to emotional expression, such as "explosive" balloons conveying high emotional arousal.
  • The authors aim to develop a system based on speech balloon generation that links voice input to emotional arousal to create corresponding balloon shapes, thereby improving emotional communication in text chats.

Solution

• Methodology and Solution:

  • The proposed EmoBalloon system uses machine learning to automatically detect emotional arousal levels from the sender's voice and generates matching "speech balloons" to enhance the emotional expression of text messages.
  • The system trains a model using Auxiliary Classifier Generative Adversarial Networks (ACGAN) with training data derived from speech balloon shapes extracted from the Manga109 dataset.
  • The system supports both a voice-input version and a manual selection version of speech balloons.

• Innovative Features:

  • Automated generation of speech balloons that match emotional arousal levels, providing a unique non-verbal visual cue.
  • Leveraging ACGAN to generate diverse speech balloon images based on continuous emotional arousal values.
  • Unlike emoticons, this approach focuses on conveying "emotional arousal intensity" rather than emotional valence (positive or negative emotions).

• Implementation Steps:

  • Extract speech balloons from the Manga109 dataset and perform emotional analysis on them.
  • Use Adobe Illustrator to interpolate balloon shapes, generating images of balloons representing different levels of emotional arousal as training data.
  • Build and train the ACGAN model to generate speech balloons corresponding to specific emotional arousal levels based on noise input.
  • The system supports both a voice-input version and a manual selection version, with user experiments conducted to evaluate its effectiveness.

Research Outcomes

• Specific Findings:

  • Experiments show that the automatically generated speech balloons effectively convey emotional arousal levels and significantly enhance recipients' perception of the emotional intensity of text messages.
  • Compared to emoticons, EmoBalloon performs better in reducing the perception gap between senders and receivers regarding emotional arousal.
  • Recipients' perception of "emotional valence" (positive or negative evaluation of the message) was not significantly influenced by the speech balloons, unlike emoticons.

• Advantages Over Existing Solutions:

  • Improves recipients' understanding of the sender's actual emotional state while reducing errors in emotional perception.
  • Focuses on conveying emotional arousal intensity, establishing a distinct functional role compared to emoticons.

• Experimental or Evaluation Results:

  • In user experiments, both the voice-input and manual selection versions of EmoBalloon significantly enhanced recipients' perception of the emotional arousal level of messages.
  • Senders using the manual selection version were more inclined to input messages with stronger emotions, while the voice-input version served more as a "feedback" tool for the sender's emotional state.

• Limitations and Future Directions:

  • The current study only focuses on two balloon shapes (circular and explosive); future research could expand to other styles (e.g., cloud-shaped or wavy balloons).
  • The Manga109 dataset primarily features Japanese content; future studies could explore other languages and cultural contexts.
  • Combining other techniques (e.g., emotional fonts or color-coded balloons) could further optimize the transmission of emotional valence.
  • Long-term testing in natural environments (e.g., daily communication) is needed to validate the system's generalizability and practical effectiveness.

Conclusion

The EmoBalloon system centers on emotional arousal, utilizing voice detection and machine learning to generate corresponding speech balloons, offering a novel solution for socio-emotional communication in text chats. It paves the way for future research and technological applications, with the potential to significantly improve the emotional accuracy of remote communication.