From Oracular to Judicial: enhancing clinical decision making through contrasting explanations and a novel interaction protocol

DOI

Clinical Decision Support Systems (CDSS) utilizing machine learning (ML) classifiers have demonstrated substantial potential for improving diagnostic accuracy across various medical domains. However, concerns regarding automation bias, diminished sense of agency, and over-reliance on these systems remain, particularly in clinical settings where decision-making autonomy is critical. To address these challenges, we propose ``Judicial AI,'' an innovative interaction protocol aimed at reducing automation bias and preserving a sense of agency. This system presents contrasting explanations to medical professionals rather than definitive recommendations, encouraging user engagement and critical evaluation. Before adopting interaction protocols that avoid definitive recommendations, it is important to assess whether such an approach impacts diagnostic accuracy, and if so, how. This paper reports an exploratory study investigating the efficacy of a Judicial CDSS in the diagnosis of vertebral fractures from X-ray images. Sixteen medical professionals, comprising spine surgeons and radiologists, participated in the diagnosis of 18 X-ray images, which were carefully selected to represent particularly difficult and complex cases. Diagnosticians first recorded their decisions independently and then with support from the Judicial AI, which provided activation maps for opposing diagnoses. Our findings show a significant improvement in diagnostic accuracy for complex cases among experienced users (p = .045), with an overall accuracy increase of 0.24. Confidence levels also rose, particularly in the case of complex diagnoses (p = .034). However, the protocol was less beneficial for less experienced users, suggesting that cognitive load might be a limiting factor. These results suggest that Judicial AI, which frames decision-makers as the ultimate authority in the decision-making process, may be an effective tool for mitigating automation bias and preserving a sense of agency in clinical environments.