Metamorpheus: Interactive, Affective, and Creative Dream Narration Through Metaphorical Visual Storytelling

Title of the Paper

Metamorpheus: Interactive, Affective, and Creative Dream Narration Through Metaphorical Visual Storytelling

Paper Information

• Subject Area: Human-Computer Interaction, Affective Computing, Experience Design
• Keywords: Affective Computing, Creative Narration, Dreams, Human-Computer Interaction, Experience-Centered Design, Visual Metaphor, Mental Health, Creative Expression

Research Background and Problem Statement

• Identified Problems or Challenges:

  • Human emotions are often shaped by life experiences, which help construct personalized meanings. However, how to support this process in daily life and leverage technologies like generative AI to foster emotional and cognitive self-reflection remains underexplored.
  • Although artistic expression has been utilized in psychotherapy to promote mental health, tools for emotional expression accessible to ordinary individuals without an artistic background are lacking.
  • Dreams, as a vivid reflection of life experiences, hold potential for fostering self-discovery and emotional well-being but are underutilized in technological applications.

• Significance of the Problem:

  • Articulating emotional experiences through language contributes to mental health and emotional recovery.
  • Capturing dream experiences can not only help users understand themselves better but also serve as a significant intervention for mental health.
  • Providing a non-specialized, accessible way for users to express and document their emotional experiences through technology is crucial.

• Research Motivation and Related Work:

  • Inspired by the third wave of Human-Computer Interaction (HCI), which emphasizes individual life experiences, and the shift in affective computing from emotion recognition to meaning-making.
  • Existing research on creative journaling (e.g., bullet journaling) and artistic expression demonstrates the potential to help users reflect and create personalized meanings.
  • Dreams, with their unique dramatic and emotional connections, offer valuable material for self-discovery and understanding personal emotions.

Proposed Solution

• Proposed Solution:

  • Metamorpheus is an affective interface designed to help users recall and reflect on emotional experiences in dreams through creative visual storytelling.
  • The system generates metaphorical images and textual descriptions based on the emotional dynamics of dreams and allows users to customize the visual layout through the interface.

• Innovative Features:

  • Leveraging generative AI (ChatGPT and Stable Diffusion) to support users in creating visual metaphors and textual narratives, fostering emotional expression.
  • Developing an interactive user interface that combines storyline visualization and emotion-driven color filters, guiding users to construct personalized meanings.
  • Applying metaphorical creation in dream narration to capture indescribable emotions and enhance user engagement.

• Implementation Steps and Key Technologies:

  • Designing a text bubble map to distinguish between literal and metaphorical scenarios, helping users identify emotional intensity.
  • Using ChatGPT to generate metaphorical text and combining it with Stable Diffusion to create text-to-image visual metaphors.
  • Allowing users to manipulate images within the interface, adjust the size and arrangement of emotional events, and personalize the interface through color filters.

Research Outcomes

• Specific Outcomes:

  • Metamorpheus enhances users' recall of dream details, enabling deeper reflection.
  • Users found the generated visual and textual metaphors to be both accurate and creative, aiding in the expression of complex emotions.
  • The system's interactive process allowed users to relive emotional memories from dreams and express personal meanings.

• Comparison with Existing Solutions and Advantages:

  • Unlike emotion recognition statistical models, Metamorpheus enhances user autonomy, offering greater creativity and control in emotional expression.
  • The system is the first to integrate generative AI to support users in concretizing and expressing complex emotions from dreams.

• Experimental or Evaluation Results:

  • Experiments with 12 participants revealed that the system improved dream recall, emotional comfort, and motivation to share.
  • Randomly generated metaphorical images or textual descriptions served as materials for users to construct meaning.
  • Users described the experience as a process of re-living dream emotions, providing therapeutic-like positive effects without emotional burden.

• Limitations and Future Directions:

  • Most users were non-native English speakers, which may have limited their perception of generative AI due to language proficiency.
  • System evaluation relied on users' linguistic expression of their experiences, while dream experiences are often hard to articulate.
  • For professional users with artistic or design backgrounds, the system's guidance and flexibility may require further adaptation.

Conclusion and Discussion

Metamorpheus is an innovative affective interface that facilitates users' narration of emotions and experiences in dreams through human-AI co-creation. Experimental results validate its potential in promoting self-reflection and sharing. Future work could extend similar systems to personal information tracking or social sharing scenarios, exploring the broader emotional connections and meaning-making capabilities of generative AI.