How Much Decision Power Should (A)I Have?: Investigating Patients’ Preferences Towards AI Autonomy in Healthcare Decision Making

Title of the Paper

How Much Decision Power Should (A)I Have?: Investigating Patients’ Preferences Towards AI Autonomy in Healthcare Decision Making

Bibliographic Information

• Research Domain: Application of Artificial Intelligence in Healthcare Decision Support
• Keywords: Artificial Intelligence, Shared Decision Making, Patient-Centered Care, Clinical Decision Support Tools, Digital Twin, Patient-Provider Collaboration

Research Background and Problem Statement

• Identified Issues and Challenges:

  1. Although Artificial Intelligence (AI) has the potential to enhance clinical decision-making in healthcare, there is insufficient research on how patients perceive the role of AI in their medical decision-making processes.
  2. Patients’ trust and acceptance of AI interventions in healthcare decisions remain uncertain, influenced by risk perception, psychological variables, and medical needs.
  3. Existing studies predominantly focus on AI decision support tools (DSTs) designed for clinicians, while the patient perspective is often overlooked.

• Significance of the Research:

  1. Patients are the ultimate beneficiaries of healthcare services, and their perspectives should be a critical input in the design of AI applications in healthcare.
  2. If AI-assisted healthcare decision-making fails to reflect patients’ preferences or needs, it may lead to distrust in AI and negatively impact healthcare outcomes.

• Motivation and Related Work:

  1. The Human-Computer Interaction (HCI) field has explored issues related to clinicians’ cognition, psychology, and professional autonomy when using AI, but there is limited research on patients’ needs and preferences.
  2. The emphasis on shared decision-making and patient-centered care values in HCI research provides theoretical support for this study.

Proposed Solution

• Research Methods:

  • Design a fictional “probe” incorporating interactive storylines and AI prototypes to explore the impact of different levels of AI autonomy on patients’ decision-making preferences.
  • Develop three hypothetical AI prototypes (DT Calculator, DT Virtual Advisor, DT Virtual Doctor) representing low, medium, and high levels of AI autonomy.
  • Collect data through semi-structured interviews (12 participants) and online surveys (15 participants).

• Innovative Aspects of the Solution:

  1. Introduced and categorized multiple levels of AI autonomy (L0 to L3), providing a framework for studying patient-AI interactions.
  2. Combined virtual storylines to contextualize patient behavior and decision-making processes, enabling real-time feedback collection.
  3. Investigated patients’ dynamic preference changes toward AI and how design can support these changes.

• Implementation Steps and Key Techniques:

  • Definition of AI Autonomy Levels:

    1. Level 0 (No AI Involvement): Patients interact exclusively with doctors.
    2. Level 1 (Tool-Type AI): AI provides predictive information based on patient needs.
    3. Level 2 (Advisory AI): AI offers recommendations while the final decision remains with the patient.
    4. Level 3 (Authoritative AI): AI autonomously makes medical decisions, with key steps requiring patient and doctor authorization.

  • Interactive Storyline Design:

    1. Set pregnancy-related interactive scenarios (low-risk and high-risk decision contexts).
    2. Use interactive storylines and videos to demonstrate AI functionalities, prompting patient reflection and choice.
    3. Collect patients’ preferences for AI and their decision-making logic.

Research Findings

• Specific Results:

  1. Most patients prefer advisory AI (L2) in healthcare decision-making but desire simultaneous involvement of doctors.
  2. Patients’ preferences for AI vary significantly depending on the risk level of the decision: low-risk scenarios favor tool-type AI, while high-risk scenarios favor advisory AI.
  3. Patients’ trust in AI is closely related to their personal health history, psychological state, and decision-making attitudes.

• Advantages:

  1. Provides profound insights into patients’ preference patterns and dynamic changes in AI-assisted healthcare decision-making.
  2. Identifies key drivers of patient preferences, offering valuable references for designing AI decision support systems with high patient acceptance.

• Experimental or Evaluation Results:

  • Data results indicate that in high-risk healthcare decision scenarios, patients are more inclined toward advisory AI (DT Virtual Advisor), whereas in low-risk scenarios, tool-type AI (DT Calculator) is more acceptable.
  • Highlights patients’ dynamic and personalized preferences, emphasizing the need for adaptability and transparency in AI systems.

• Limitations and Future Directions:

  1. The sample size is relatively small, and further research is needed to validate the generalizability of the findings with larger samples.
  2. Cultural differences may influence perceptions of AI roles, warranting exploration of cross-cultural applicability.
  3. The study focuses on female patients’ preferences for AI, but the opinions of family members or other stakeholders may significantly impact final decisions.
  4. Suggests exploring how AI can dynamically meet patients’ needs and preferences from the perspective of long-term use and sustained trust.

This study highlights the potential and design challenges of AI as a healthcare decision support tool, emphasizing the importance of truly reflecting patient-centered values rather than merely expanding technical functionalities. It provides critical theoretical foundations for the future design and deployment of AI healthcare technologies.