"When Two Wrongs Don't Make a Right" - Examining Confirmation Bias and the Role of Time Pressure During Human-AI Collaboration in Computational Pathology

Research Background and Issues

Problems or Challenges Identified by the Authors

1. Amplification of Cognitive Biases: In Clinical Decision Support Systems (CDSS) based on Artificial Intelligence (AI), AI may exacerbate cognitive biases, such as confirmation bias. When an initial human judgment is incorrect and the AI provides similarly erroneous suggestions, this bias manifests as a false confirmation, reinforcing the erroneous judgment.
2. Complexity of Time Pressure: Time pressure is prevalent in medical settings and may further amplify cognitive biases, as decision-makers tend to rely more heavily on system recommendations when cognitive resources are limited.
3. Research Gap in Sequential Decision-Making: Most studies in the medical domain focus on discrete decision-making, whereas many routine medical tasks, such as estimating tumor cell percentages in pathology, involve sequential decision-making.

Importance of the Problem

In high-risk domains such as healthcare, diagnostic errors can have direct life-or-death consequences for patients' well-being and treatment plans. The reinforcement of erroneous judgments (e.g., alignment between AI and expert mispredictions) can magnify the severity of diagnostic errors, undermining the potential benefits of AI-assisted systems.

Research Motivation and Related Work

1. Significance of Cognitive Biases: Cognitive biases, such as confirmation bias, are well-documented phenomena in medical diagnosis. However, the risks and dynamics introduced by AI have not been fully understood.
2. Potential Role of Time Pressure: While time pressure is ubiquitous in daily medical practice, few studies have explored how it influences confirmation bias induced by AI.
3. Addressing the Gap in Sequential Decision-Making Contexts: Existing research on AI and confirmation bias has primarily focused on discrete decisions. This study aims to fill the gap in understanding confirmation bias in sequential decision-making scenarios.

Proposed Solution

Methods or Solutions Proposed by the Authors

1. Conduct web-based experiments simulating the process of pathologists estimating tumor cell proportions under different conditions (with/without AI support, with/without time pressure).
2. Use linear mixed-effects models to quantitatively analyze the relationship between AI recommendations and expert judgments, specifically evaluating the presence of confirmation bias.
3. Investigate how time pressure influences confirmation bias and the extent to which it affects reliance on AI-assisted tasks.

Innovations of the Solution

1. Data-Driven Analysis: For the first time, linear mixed-effects models are used to quantify the presence and variation of confirmation bias in sequential tasks.
2. New Insights on Time Pressure: Exploration of time pressure as a critical variable reveals its complex interaction with AI and human experts.
3. Domain-Specific Tasks: By conducting direct experiments on significant tasks in pathology, such as tumor cell proportion estimation, the study enhances the practical applicability and external validity of its conclusions.

Implementation Steps and Key Techniques

1. Experimental Design:
   • Two-phase task: Participants first independently estimate the tumor cell proportion (TCP) and then re-estimate after being provided with AI assistance.
   • Four conditions: independent decision-making vs. AI-supported; with time pressure vs. without time pressure.
   • A two-week washout period is implemented to mitigate memory effects on image recognition.
2. Data Collection and Analysis:
   • Record experts' evaluation values in experimental scenarios and analyze their adjustment paths under AI recommendations.
   • Use paired samples and linear mixed-effects models to examine the strength of associations between variables.
3. Evaluation of Confirmation Bias:
   • Select samples with erroneous judgments and inaccurate AI results.
   • Perform weighted analysis by categorizing AI suggestions as consistent/inconsistent.

Research Outcomes

Specific Findings

1. Evidence of Confirmation Bias:
   • When AI algorithm suggestions align with experts' independent judgments (even if both are incorrect), experts are more likely to accept the AI's recommendations.
   • Confirmation bias is particularly pronounced in scenarios where both experts and AI make aligned errors.
2. Impact of Time Pressure:
   • Interestingly, time pressure did not exacerbate confirmation bias but instead weakened its manifestation. The study shows that under time pressure, participants tended to rely unconditionally on AI recommendations, potentially linked to "automation bias."
3. Applicability to Sequential Decision-Making Tasks:
   • The study is the first to demonstrate the presence of AI-induced confirmation bias in sequential decision-making tasks involving quantitative assessments.

Advantages Compared to Existing Solutions

• Quantifies the impact of cognitive biases under different conditions (AI consistency, time pressure), providing quantitative insights for designing safer AI decision-support tools.
• Utilizes real pathology tasks with medical expert participation, enhancing the external validity and clinical relevance of the study.

Experimental or Evaluation Results

1. Analysis using linear mixed-effects models reveals:
   • Confirmation bias is more pronounced when AI predictions align with expert judgments (positive regression coefficient).
2. The effect of "automation bias" under time pressure indicates:
   • Increased reliance on AI recommendations by experts (Judge Adviser System mean value increased from 0.49 to 0.55 under time pressure).
3. Detailed experimental data and statistical descriptions of participant behavior are provided:

Limitations and Future Directions

1. Sample Size: The limited number of 28 participants may affect the generalizability of the study results.
2. Simulation of Time Pressure: The countdown for individual tasks in the experimental conditions does not fully replicate the high workload pressure in real clinical settings.
3. Future Research Directions:
   • Explore mechanisms of confirmation bias in diverse medical tasks, such as pathology imaging analysis or drug selection.
   • Introduce bias mitigation strategies (e.g., workflow guidance tools or cognitive forcing functions) and evaluate their effectiveness.
   • Validate findings with larger sample sizes to enhance the generalizability of results.

In summary, this study not only reveals the specific manifestations of confirmation bias in AI-assisted medical decision-making but also provides valuable insights and empirical foundations for the future development of safe and efficient clinical decision-support tools. Moreover, the complex interaction between time pressure and AI assistance presented in the study points to new directions for in-depth research on cognitive biases.