Who Tells the Better Story? Comparing Human and AI-Generated Narratives on Perinatal Mental Health Topics

Paper Title

Who Tells the Better Story? Comparing Human and AI-Generated Narratives on Perinatal Mental Health Topics

Publication Info

• Topic area: Human-AI interaction in storytelling for perinatal mental health.
• Keywords: Perinatal mental health, storytelling, AI-generated narratives, human-authored stories, emotional catharsis, narrative evaluation, cultural resonance, mixed-methods study, GPT-4, peer support.

Background and Problem

• Problem / challenge: AI-generated narratives are increasingly used in mental health contexts, but their ability to match the authenticity, cultural nuance, and emotional resonance of human-authored stories remains unclear.
• Significance: Understanding how AI narratives compare to lived-experience stories is crucial for designing effective and trustworthy storytelling technologies in sensitive domains like perinatal mental health.
• Motivation and related work: Prior research highlights the importance of storytelling in reducing stigma, fostering connection, and supporting emotional processing in perinatal contexts. While AI storytelling has shown promise in other health domains, it often lacks cultural nuance and emotional depth, raising concerns about its applicability in perinatal care.

Solution

• Proposed approach: A mixed-methods study comparing AI-generated and human-authored narratives on perinatal mental health topics, focusing on believability, engagement, relevance, and emotional catharsis.
• Novelty:
  1. Empirical comparison of AI- and human-authored narratives across key storytelling dimensions.
  2. Integration of human evaluations, qualitative reflections, and automated NLP-based analyses to assess narrative quality.
  3. Design implications for human-centered storytelling technologies that foreground lived experiences while cautiously incorporating AI.
• Procedure and key techniques:
  • Data collection: 20 AI-generated stories (via GPT-4) and 20 human-authored stories sourced from perinatal support forums, covering eight thematic categories.
  • Participant evaluation: 400 female-identifying participants in perinatal stages rated stories on a 7-point Likert scale across four constructs (believability, engagement, relevance, catharsis).
  • Automated analysis: NLP metrics assessed readability, coherence, sentiment, and emotional arcs.
  • Mixed-methods analysis: Quantitative comparisons, qualitative thematic analysis, and alignment between human and automated evaluations.

Results

• Concrete findings:
  • Both AI- and human-authored stories received high ratings (≈5.8–6.2 on a 7-point scale).
  • Human-authored stories were rated slightly higher on engagement, relevance, and emotional catharsis, while AI stories scored marginally better on completeness and coverage.
  • Emotional catharsis was linked to emotional arcs (e.g., sadness to hope) rather than overall positivity.
• Advantage over baselines:
  • Human-authored stories excelled in emotional authenticity, cultural nuance, and interpersonal dynamics.
  • AI narratives were appreciated for structural coherence but critiqued for emotional distance and generic cultural framing.
• Experiments / evaluation:
  • Participants evaluated stories across eight themes (e.g., trauma, isolation, breastfeeding struggles) and three perinatal stages (pregnancy, postpartum, both).
  • Automated NLP metrics underestimated human ratings and failed to capture cultural and emotional depth.
• Limitations and future work:
  • Limited cultural diversity in the participant pool (U.S.-based MTurk workers).
  • Automated metrics lacked sensitivity to relational and cultural nuances.
  • Future work should explore longitudinal impacts, inclusive datasets, and participatory design approaches.

Summary

This study compared AI-generated and human-authored narratives on perinatal mental health topics, revealing complementary strengths. Human-authored stories were valued for their emotional authenticity, cultural resonance, and interpersonal depth, while AI narratives offered structural coherence but lacked emotional nuance. Automated analyses underestimated narrative quality, highlighting the need for human-centered evaluation frameworks. The findings suggest that AI storytelling should play a secondary, optional role, supporting but not replacing lived-experience narratives. Design implications include creating digital storybooks that foreground community-authored content while cautiously integrating AI for structural tasks. Future research should address cultural inclusivity, long-term effects, and trust dynamics in AI-assisted storytelling.