Framing Responsible Design of AI for Mental Well-Being: AI as Primary Care, Nutritional Supplement, or Yoga Instructor?

Paper Title

Framing Responsible Design of AI for Mental Well-Being: AI as Primary Care, Nutritional Supplement, or Yoga Instructor?

Publication Info

• Topic area: Responsible design and evaluation of AI tools for mental well-being.
• Keywords: AI ethics, mental well-being, large language models, responsible design, health policy, digital therapeutics, user safety, primary care, nutritional supplements, yoga analogy.

Background and Problem

• Problem / challenge: Non-clinical LLM tools like ChatGPT are increasingly used for mental well-being, but their benefits and risks are poorly understood, and there is no clear framework for their responsible design and evaluation.
• Significance: These tools could alleviate mental healthcare shortages but also pose risks such as delaying clinical care, fostering oversimplified views of mental health, and exacerbating inequalities.
• Motivation and related work: Prior research has identified potential benefits and harms of these tools but lacks actionable frameworks for responsible design. Current industry efforts, such as FDA approval or detection-and-warning features, have proven inadequate or impractical.

Solution

• Proposed approach: A framework for responsible design of non-clinical LLM tools, using analogies to nutritional supplements, over-the-counter drugs, yoga instructors, and primary care providers to clarify responsibilities and risks.
• Novelty:
  1. Introduces analogies to frame the responsibilities of LLM tools based on their guaranteed benefits and intended users.
  2. Proposes three criteria for responsible design: specific guaranteed benefits, effective delivery of active ingredients, and commensurate risks and benefits.
  3. Highlights distinct risks and responsibilities for different types of LLM tools.
  4. Offers tailored evaluation criteria for various LLM tool designs.
• Procedure and key techniques:
  • Conducted 24 expert interviews across AI ethics, health policy, and clinical domains.
  • Analyzed over 100 policy documents to contextualize findings.
  • Refined findings through additional expert deliberations.

Results

• Concrete findings:
  • Responsible design depends on the tool’s guaranteed benefits (e.g., general well-being vs. specific symptom relief) and its ability to deliver proven active ingredients.
  • Tools analogous to primary care or over-the-counter drugs must prioritize safety, effectiveness, and equitable access.
  • Tools analogous to nutritional supplements or yoga instructors must prevent misuse, overuse, and displacement of clinical care or essential self-care.
• Advantage over baselines:
  • Provides actionable criteria for responsible design, addressing gaps in prior frameworks.
  • Clarifies the distinct responsibilities of tool creators, model developers, and users.
• Experiments / evaluation:
  • Expert interviews revealed consensus on the importance of articulating benefits, ensuring effective delivery, and balancing risks and benefits.
  • Policy analysis identified transferable regulatory principles from healthcare, supplements, and yoga instruction.
• Limitations and future work:
  • Disagreements among experts on acceptable risk levels and the responsibility of nutritional-supplement-like tools.
  • Future research should explore higher standards for responsible design and evaluate tools from public health and rights-based perspectives.

Summary

This paper proposes a framework for the responsible design of non-clinical LLM tools for mental well-being, using analogies to supplements, drugs, yoga instructors, and primary care providers. It highlights three criteria: specific guaranteed benefits, effective delivery of active ingredients, and commensurate risks and benefits. Findings are based on expert interviews and policy analysis, offering actionable guidance for designers and regulators. Future work should address unresolved debates on risk evaluation and explore innovative design and regulatory strategies.