Principles of Safe AI Companions for Youth: Parent and Expert Perspectives

Paper Title

Principles of Safe AI Companions for Youth: Parent and Expert Perspectives

Publication Info

• Topic area: Youth safety in interactions with generative AI companions.
• Keywords: AI companions, youth safety, developmental psychology, parental mediation, risk assessment, generative AI, conversational agents, online safety, stakeholder perspectives, intervention design.

Background and Problem

• Problem / challenge: Current AI companions lack sufficient safeguards to address developmental risks and harmful interactions with youth, such as emotional dependence, inappropriate content, and normalization of harmful norms.
• Significance: Youth are particularly vulnerable due to their developmental stage, and harmful interactions with AI companions could shape their social, emotional, and moral development.
• Motivation and related work: Prior research highlights risks like sexual harassment, emotional dependence, and blurred boundaries in AI-human interactions, but lacks empirical studies on how parents and experts assess these risks or what safeguards they recommend. This paper addresses this gap by involving these stakeholders in evaluating real-world youth–AI interactions.

Solution

• Proposed approach: Conducted 26 semi-structured interviews with parents and developmental psychology experts to assess risks, benefits, and interventions for youth–AI interactions.
• Novelty:
  1. First empirical study using real conversational data to elicit multi-stakeholder perceptions of youth–AI interactions.
  2. Identification of layered contextual factors and differing logics of risk assessment between parents and experts.
  3. Stakeholder-suggested principles and interventions for system design, interaction safeguards, and social involvement.
• Procedure and key techniques:
  • Collected 253 real-world youth–AI conversation logs from Character.ai and selected 8 diverse snippets for review.
  • Conducted interviews with 8 parents and 13 experts, focusing on perceived risks, benefits, and intervention strategies.
  • Thematic analysis of interview transcripts using a hybrid inductive-deductive coding approach.

Results

• Concrete findings:
  • Parents flagged single events (e.g., mentions of suicide, flirtation) as high risk, while experts focused on patterns over time (e.g., repeated self-harm references, emotional dependence).
  • Both groups identified contextual factors such as youth age, AI character age, interaction frequency, and AI-modeled behaviors as critical in assessing risks.
  • Benefits included providing a safe space for self-expression, practicing social skills, and learning healthy boundaries.
• Advantage over baselines: Provides nuanced, multi-stakeholder insights into youth–AI interactions, surpassing prior studies that relied on hypothetical scenarios or single-stakeholder perspectives.
• Experiments / evaluation:
  • Participants reviewed anonymized conversation snippets and provided feedback on risks, benefits, and desired interventions.
  • Analysis revealed distinct logics of risk assessment and intervention preferences between parents and experts.
• Limitations and future work:
  • Lack of direct youth perspectives; future studies should include youth voices.
  • Small, U.S.-based sample limits generalizability; future research should expand to global and culturally diverse contexts.
  • Qualitative focus; future work could incorporate mixed methods or longitudinal studies.

Summary

This study explores how parents and developmental psychology experts assess the risks and benefits of youth interactions with AI companions, using real-world conversational data. Parents emphasized immediate risks and value alignment, while experts focused on developmental skill acquisition and patterns of harm. Both groups advocated for layered safeguards, including system-level transparency, interaction-level monitoring, and context-aware interventions. Findings highlight the need for personalized, developmentally informed, and family-sensitive AI designs that balance youth autonomy with safety. Future research should incorporate youth perspectives and test proposed interventions in broader contexts.