``It Is a Moving Process'': Understanding the Evolution of Explainability Needs of Clinicians in Pulmonary Medicine

Title of the Paper

“It Is a Moving Process”: Understanding the Evolution of Explainability Needs of Clinicians in Pulmonary Medicine

Paper Information

• Research Domain: Application of artificial intelligence in healthcare, particularly the explainability of AI-assisted Clinical Decision Support Systems (CDSS)
• Keywords: Explainable Artificial Intelligence (XAI), Clinical Decision Support Systems (CDSS), Medical AI, User Needs, Information Dynamics, Pulmonary Medicine, Medical Education

Research Background and Issues

• Identified Problems/Challenges:

  1. Current explainability research primarily focuses on static (single-point) issues, neglecting the temporal dynamics of medical processes.
  2. Traditional XAI studies are often technology-centric, paying less attention to specific user needs (e.g., clinicians) and how these needs evolve during patient care.
  3. Medical practice is highly complex, requiring clinicians to make decisions in dynamic and uncertain environments, which imposes higher demands on the explanatory capabilities of AI-assisted systems.

• Significance of the Study:

  • In high-risk and high-pressure environments like healthcare, explainability (XAI) can help clinicians better trust and utilize AI systems, ensuring decision reliability and transparency.
  • By deeply understanding the evolving needs of clinicians for AI system explainability during clinical processes, the design of clinical decision support systems can be significantly improved to better align with real-world demands.

• Motivation and Related Work:

  • The authors reviewed prior research on XAI algorithms and human factors, noting that these studies often overlook the dynamic changes in clinicians' needs during medical processes.
  • Specifically, in cases of certain diseases (e.g., Idiopathic Pulmonary Fibrosis, IPF), the dynamic evolution of clinicians' information needs due to changes in patient conditions and treatments has not been adequately studied.

Proposed Solution

• Methods and Approach:

  • An exploratory research method was proposed, using patient journey mapping to model the evolution of clinicians' information needs during IPF care.
  • The study was conducted in two phases: Phase 1 involved collaborative sessions to explore a needs framework; Phase 2 involved semi-structured interviews with clinicians to further validate and analyze these needs.

• Innovations of the Study:

  1. Focus on dynamic rather than static explainability needs, addressing gaps in previous research.
  2. Integration of a human-centered perspective into XAI design and evaluation, emphasizing the temporal dimension and contextual relevance of user needs.
  3. Introduction of patient journey mapping as a research framework to support understanding of the evolution of clinicians' explainability needs.

• Implementation Steps and Key Techniques:

  1. Patient Journey Modeling: Data-driven mapping of patient community journeys based on IPF patient stories from the U.S. online community Inspire and clinical pathways from the Netherlands' Erasmus Medical Center.
  2. Exploratory Interviews: Collaborative design of an ideal patient journey map for IPF with clinicians and researchers from multidisciplinary backgrounds.
  3. Application of XAI Explanation Examples: Preparation of various explanation formats (e.g., images, numbers, text, rules) to help interviewed clinicians reflect on their needs and stimulate discussion.
  4. Qualitative Analysis: Thematic analysis of interview content to identify trends in dynamic needs and propose design recommendations for XAI design and education.

Research Outcomes

• Specific Findings:

  1. Identification of clinicians' explainability needs during IPF care, including:
     • General Needs: Non-technical explanations about the capabilities and limitations of AI systems, particularly key information about training data and validation standards.
     • Localized Patient-Specific Needs: Explanations tailored to the specific circumstances of individual patients (e.g., test results and comorbidities).
     • Multimodal Needs: Preference for visualized, multimodal explanations that integrate multiple sources of information.
  2. Design insights for XAI in medical contexts, emphasizing interactive explanatory mechanisms that support long-term learning and gradual adaptation.

• Advantages:

  • Demonstrated that dynamic, contextualized explanations are better suited to supporting clinicians' practical needs compared to static explanations.
  • Promoted a paradigm shift in XAI from technology-centric to human-centered approaches, emphasizing clinicians' agency in complex medical workflows.

• Experimental/Evaluation Results:

  1. Clinicians' needs gradually transitioned from "general explanations" (understanding AI system capabilities in phases) to "patient-specific explanations" (focused on specific guidance for diagnosis and treatment).
  2. Most clinicians acknowledged the benefits of multimodal explanations, such as text-visual combinations that are easier to understand and actionable.
  3. The perceived importance of explanations diminished as clinicians became more familiar with the AI system.

• Limitations and Future Directions:

  • Limitations:

    1. The study was limited to IPF and may not directly apply to other diseases.
    2. The research sample was drawn from a single European university medical center, introducing regional limitations.
    3. The provided explanation examples might have introduced anchoring bias during interviews.

  • Future Directions:

    1. Validate findings in different diseases and medical contexts, such as cancer and chronic illnesses.
    2. Explore interactivity and variability in explanation design, such as customizable multi-level explanations.
    3. Promote AI literacy in medical education, including interdisciplinary collaboration to bridge the knowledge gap between AI and medicine.