Large Language Models in Peer-Run Community Behavioral Health Services: Understanding Peer Specialists and Service Users’ Perspectives on Opportunities, Risks, and Mitigation Strategies

Paper Title

Large Language Models in Peer-Run Community Behavioral Health Services: Understanding Peer Specialists and Service Users’ Perspectives on Opportunities, Risks, and Mitigation Strategies

Publication Info

• Topic area: Integration of large language models (LLMs) into peer-run behavioral health services.
• Keywords: Large language models, peer-run organizations, behavioral health, relational authority, trust, lived experience, community-centered AI, mitigation strategies, co-design, recovery-oriented care.

Background and Problem

• Problem / challenge: Peer-run organizations (PROs) face systemic challenges such as underfunding, limited technological infrastructure, and growing demand, which hinder their ability to scale and improve service delivery. The introduction of LLMs into these settings raises concerns about contextual misalignment, relational erosion, and the displacement of experiential authority.
• Significance: PROs provide critical, recovery-oriented support to underserved populations, addressing behavioral health disparities. Ensuring that LLMs align with the values and practices of peer support is essential to preserving the relational authority and trust that define these services.
• Motivation and related work: While LLMs have been applied in behavioral health contexts, prior systems often lack community engagement and fail to address the unique needs of PROs. Existing research in human-computer interaction (HCI) has explored community-centered AI design but has rarely focused on peer-run behavioral health settings, leaving a gap in understanding how LLMs intersect with recovery-oriented peer practices.

Solution

• Proposed approach: The study explores how LLMs can be integrated into PROs by centering lived experience, relational authority, and community-grounded design. It uses comicboarding as a co-design method to engage peer specialists and service users in identifying opportunities, risks, and mitigation strategies.
• Novelty:
  1. Empirical insights into how frontline stakeholders perceive LLM integration in peer support workflows.
  2. Introduction of "lived-experience-in-the-loop" as a design principle for LLM-supported care.
  3. Actionable design implications for aligning LLMs with recovery-oriented, community-led practices.
• Procedure and key techniques:
  • Partnered with Collaborative Support Programs of New Jersey (CSPNJ) to conduct workshops with 16 peer specialists and 10 service users.
  • Used comicboarding to illustrate LLM capabilities, limitations, and improvement pathways.
  • Explored three tensions: scale vs. situatedness, trust vs. mistrust, and autonomy vs. automation.
  • Analyzed workshop transcripts using reflexive thematic analysis to identify themes and design implications.

Results

• Concrete findings:
  • LLMs can sustain, undermine, or amplify relational authority depending on their implementation.
  • Participants identified three key tensions: bridging scale and local knowledge, protecting trust, and balancing autonomy with automation.
  • Opportunities include improving resource navigation, supporting holistic guidance, and streamlining workflows.
  • Risks include context-blind recommendations, erosion of trust, and over-reliance on automation.
• Advantage over baselines: The study emphasizes relational authority and lived experience as central to LLM design, moving beyond traditional human-in-the-loop frameworks to prioritize community-grounded and recovery-oriented practices.
• Experiments / evaluation:
  • Conducted nine workshops and two individual interviews using comicboarding.
  • Participants reflected on LLM opportunities, risks, and mitigation strategies across three lifecycle stages.
  • Data analysis revealed themes related to relational authority and its preservation.
• Limitations and future work:
  • Lack of hands-on interaction with live LLM systems may have limited ecological validity.
  • Study focused on a single PRO in one U.S. state; findings may not generalize to other contexts.
  • Future work should include live demonstrations, interactive prototypes, and evaluations of comicboarding as a co-design method.

Summary

This study investigates the integration of large language models (LLMs) into peer-run behavioral health services, emphasizing the importance of relational authority and lived experience. Through workshops with peer specialists and service users, the research identifies opportunities (e.g., enhancing resource navigation and streamlining workflows), risks (e.g., context-blind guidance and trust erosion), and mitigation strategies (e.g., peer-led alignment and lived-experience-in-the-loop design). The findings highlight the need for community-centered AI that preserves peer autonomy and trust while amplifying relational care. These insights offer actionable guidance for responsibly deploying LLMs in high-stakes, community-led settings.