Words to Describe What I’m Feeling: Exploring the Potential of AI Agents for High Subjectivity Decisions in Advance Care Planning

Paper Title

Words to Describe What I’m Feeling: Exploring the Potential of AI Agents for High Subjectivity Decisions in Advance Care Planning

Publication Info

• Topic area: AI-enhanced decision-making in Advance Care Planning (ACP)
• Keywords: Advance Care Planning, AI proxy, high-subjectivity decisions, decision support, agent autonomy, human control, patient advocacy, generative AI, end-of-life planning, healthcare technology

Background and Problem

• Problem / challenge: Traditional Advance Care Planning (ACP) relies on human proxies, but demographic shifts have reduced caregiver availability, and existing ACP practices are static, often failing to adapt to evolving clinical and personal circumstances. AI's potential role in high-risk, high-subjectivity decisions like ACP remains underexplored.
• Significance: Addressing gaps in ACP practices is critical as societies age and caregiver availability declines. AI systems could provide scalable, adaptable support for representing patient values in critical medical decisions.
• Motivation and related work: Prior research has focused on AI for extracting preferences, predicting ACP eligibility, and guiding value exploration. However, these tools remain advisory and lack the capacity to act as autonomous proxies. This study builds on this gap by exploring AI's role in high-risk, subjective decision-making contexts.

Solution

• Proposed approach: Development and evaluation of ACPAgent, an AI-powered prototype designed to simulate training an agent to act as a personal proxy for resuscitation decisions in ACP.
• Novelty:
  1. Introduction of ACPAgent as a trainable AI proxy for high-risk, high-subjectivity decisions.
  2. Empirical findings on user perceptions and interactions with AI in ACP contexts.
  3. Design considerations for AI as a patient advocate, bridging gaps in current ACP practices.
• Procedure and key techniques:
  • Participants trained ACPAgent across five CPR scenarios, providing preferences and feedback.
  • Scenarios covered a range of health trajectories, from recoverable episodes to advanced decline.
  • Participants reflected on ACPAgent's recommendations, adjusted preferences, and envisioned its potential roles.
  • Data were analyzed using thematic coding and mapped onto Shneiderman’s autonomy-control framework.

Results

• Concrete findings:
  • Participants agreed with ACPAgent’s recommendations in 86.7% of cases.
  • Scenario details influenced decisions in 76% of cases, highlighting the importance of contextual factors like prognosis, independence, and financial burden.
  • Participants envisioned four roles for ACPAgent: decision-support tool, educational tool, legal proxy, and proxy with human sign-off.
• Advantage over baselines: ACPAgent provided a reflective, interactive platform for exploring values and decisions, unlike static ACP documentation. It also supported bi-directional learning, where participants and the agent influenced each other.
• Experiments / evaluation:
  • Conducted with 15 participants across four workshops.
  • Participants engaged with five CPR scenarios, adjusted preferences, and provided feedback on ACPAgent’s recommendations.
  • Data included preference changes, agreement rates, and qualitative reflections.
• Limitations and future work:
  • Simplified scenarios and single-session workshops limited ecological realism.
  • The interface lacked multi-turn dialogue and reasoning transparency.
  • Future work should explore longitudinal deployments, ambiguous cases, and cultural influences on AI in ACP.

Summary

This study introduced ACPAgent, an AI prototype designed to act as a proxy in Advance Care Planning (ACP). Across five CPR scenarios, participants trained ACPAgent and reflected on its recommendations, revealing high agreement rates (86.7%) and diverse envisioned roles, including decision-support tool, educational tool, legal proxy, and proxy with human sign-off. The study highlighted the potential of AI as a patient advocate, capable of amplifying and reinforcing values while complementing human oversight. Future work should address limitations in scenario realism, interaction design, and regulatory frameworks to advance AI's role in high-risk, high-subjectivity decision-making contexts.