"When He Feels Cold, He Goes to the Seahorse"—Blending Generative AI into Multimaterial Storymaking for Family Expressive Arts Therapy

Title of the Paper

"When He Feels Cold, He Goes to the Seahorse"—Blending Generative AI into Multimaterial Storymaking for Family Expressive Arts Therapy

Paper Information

• Research Area: Application of HCI and generative AI in family arts therapy
• Keywords: Expressive arts therapy, storymaking, family, children, generative AI, HCI

Research Background and Problem Statement

• Identified Problems or Challenges:

  • Family communication is crucial for parent-child relationships and children's psychological development, but it is often hindered by technology dependence, family conflicts, and poor communication patterns.
  • While storymaking is widely used in family expressive arts therapy, few studies explore how generative AI can serve as an expressive material in such therapeutic settings.
  • The current HCI field lacks design practices that support families and therapists in storymaking within the context of arts therapy.

• Significance:

  • Family expressive arts therapy enhances communication among family members and has long-term benefits for children's mental health and overall development.
  • Emerging technologies like generative AI may lower creative barriers in certain contexts, providing new materials for family expression.

• Research Motivation and Related Work:

  • The integration of traditional materials with generative AI in storymaking remains in the exploratory stage.
  • Previous HCI research has examined the participation of children and families in storymaking but lacks studies supporting multimaterial creation in therapeutic contexts.

Proposed Solution

• Method or Solution:

  • Propose using generative AI as an expressive material in arts therapy, combined with traditional physical creation materials.
  • Conduct case studies to examine how families engage in AI-based multimaterial storymaking under the guidance of a therapist.

• Innovations:

  • Innovatively combine generative AI materials with handcrafted creations by children and parents to create rich, expressive, and therapeutic art experiences.
  • Introduce the "Expressive Therapies Continuum" (ETC) framework to analyze the therapeutic significance of generative AI materials.

• Implementation Steps and Key Techniques:

  1. Introduce generative AI tools (e.g., Midjourney) into real-world family expressive arts therapy, alongside traditional materials such as colored paper, clay, fabric, and LEGO.
  2. Use family-made crafts and textual descriptions as input for AI-generated images, guiding families to select suitable outputs and transform them into physical materials.
  3. Combine these materialized AI images with traditional materials for further creation, culminating in expressive outcomes such as storybooks.
  4. Collect process records and interview data from families, and invite expert therapists for analysis and validation.

Research Outcomes

• Specific Findings:

  • Documented the participation processes and actions of children, parents, and therapists in multimaterial storymaking. Demonstrated that generative AI can empower family arts therapy practices by lowering creative barriers and enriching expression.
  • Showed that integrating generative AI with physical material creation enhances the creative experience across multiple dimensions, such as emotional expression, cognitive planning, and symbolic self-expression.

• Advantages Compared to Existing Solutions:

  • Overcomes the expressive and creative limitations of traditional art materials, providing families with high-quality and diverse artistic outputs.
  • Bridges the gap between digital and physical materials in therapeutic interactions, extending virtual creations into real family life.

• Experimental or Evaluation Results:

  • In experiments involving 18 participants from different families (including children and parents), all successfully utilized AI and traditional materials for storymaking.
  • Qualitative analysis revealed that AI materials facilitated interactions between children and parents, uncovered new aspects of family members, and enhanced mutual understanding within families.
  • The variety of options provided by generative AI encouraged active participation from both parents and children during multiple rounds of creation.

• Limitations and Future Directions:

  • The unpredictability of AI outputs may lead to results that deviate from expectations, necessitating more user-friendly input interfaces and preprocessing mechanisms.
  • Suggest further development of interactive design solutions that support role-playing, family communication, and co-construction of meaning, exploring how AI-created artifacts can serve as transitional objects in family life.
  • Advocate for the implementation of safe and age-appropriate AI principles to minimize potential risks for children while fostering positive creative behaviors.