Rethinking the Role of AI with Physicians in Oncology: Revealing Perspectives from Clinical and Research Workflows

Title of the Paper

Rethinking the Collaborative Role of Artificial Intelligence in Oncology: Perspectives from Clinical and Research Workflows

Document Information

• Subject Area: The role of artificial intelligence in oncology and human-computer collaboration
• Keywords: AI applications in oncology, clinical AI adoption, imaginaries, explainability, contestability, human-centered AI, closed-loop human-computer collaboration

Research Background and Problem

• What issues or challenges did the authors identify?

  • Despite significant advancements in artificial intelligence (AI) for cancer research, its impact on clinical applications remains very limited.
  • One of the root causes is that current AI development does not sufficiently consider the fundamental differences between clinical and research practices, particularly in terms of intent and scope.
  • Clinicians' trust in AI is more influenced by their contestable interactions with AI rather than a general acceptance of AI.

• Why is this issue important?

  • The advancement of precision medicine and personalized treatment heavily relies on AI technology but is hindered by low clinical adoption rates.
  • The current "one-size-fits-all" AI development model fails to meet the specific needs and contextual adaptability of clinical workflows, obstructing its deeper application in clinical treatment.

• Research Motivation and Related Work

  • Conduct a detailed investigation into the role of AI in oncology's clinical and research workflows and clinicians' expectations of AI to address critical gaps in human-computer collaboration and medical AI development.
  • Related studies highlight the broad potential of medical imaging AI but point out challenges such as the black-box effect, limited sample sizes, and ethical and accountability issues, which prevent large-scale clinical implementation.
  • Emphasize the importance of interdisciplinary collaboration between HCI (human-computer interaction) and AI to address these challenges.

Proposed Solution

• What methods or solutions did the authors propose?

  • Conduct a semi-structured interview study with seven physicians engaged in cancer treatment and research to explore the current state, expectations, and challenges of AI applications in oncology.
  • Analyze the clinical and research practices of these physicians to identify the inherent tensions between the two workflows and the role and positioning of AI within them.
  • Introduce human-centered and closed-loop human-computer collaboration design principles to recalibrate the development and deployment pathways of AI technologies in oncology.

• What is innovative about this solution?

  • Proposed understanding the significance of AI to clinicians and medical workflows through the lens of "imaginaries," a sociological approach that offers a novel perspective on defining AI's societal role.
  • Systematically examined issues of explainability and contestability in medical contexts, highlighting their differentiated needs and practical implications in clinical decision support systems (CDSS).

• What are the implementation steps and key technical points?

  1. Data Collection: Conduct one-on-one interviews with seven physicians (specializing in nuclear medicine, radiology, oncology, etc.) with clinical and research experience.
  2. Interview Topics include:
     • Physicians' professional backgrounds and workflows.
     • Interaction with patients and decision-making processes in cancer treatment.
     • Current perceptions of AI and expectations for its future role.
  3. Data Analysis Methods: Perform open coding and axial coding on the interview content to identify recurring themes and common issues.
  4. Develop two conceptual models for AI applications:
     • Tool-based (as a decision-support tool).
     • Constitutive (as an equal participant in tumor board meetings).

Research Findings

• What specific findings were achieved?

  • Clinicians' trust in AI primarily depends on their contestable interactions with AI and their preference for supervised use of AI (clinician-in-the-loop model).
  • Identified three core contextual requirements for AI:
    1. Explainability: Clinicians need AI to explicitly demonstrate data sources and their connections to pathological biology.
    2. Contestability: Develop trust based on local validation and iterative processes grounded in personal experience.
    3. Collaboration: AI must seamlessly integrate into existing ecosystems of clinical decision-making meetings (e.g., multidisciplinary tumor boards, MTB).
  • AI currently plays a significantly larger role in research than in clinical practice, but the ethical and accountability challenges differ between the two workflows.

• What advantages does it have compared to existing solutions?

  • Breaks away from the traditional single-workflow perspective by combining clinical and research practices, highlighting the opposing tensions in human-centered AI design.
  • Provides pragmatic design pathways to lower barriers for clinical AI adoption (e.g., data governance and explainability).

• What are the experimental or evaluation results?

  • Clinicians' acceptance and trust in AI vary depending on the work environment (research vs. clinical), goals (long-term academic outcomes vs. individual patient treatment), and tool requirements.
  • In clinical decision-making, AI is more often perceived as an "assistant" rather than a "leader," necessitating supervised operation by clinicians.

• Limitations and Future Directions

  • Small sample size (7 physicians); future studies should involve a broader group of clinicians to validate initial findings.
  • The specific functional design of AI in simulated multidisciplinary tumor board environments has not yet been fully empirically studied; further development of prototypes for "tool-based" and "constitutive" AI is needed.
  • Recommend interdisciplinary collaboration to further deconstruct the implications of "contestability" and "explainability" in clinical and research practices.