Benefits of Machine Learning Explanations: Improved Learning in an AI-assisted Sequence Prediction Task

DOI

Research in Explainable AI (XAI) has shown that explanations can improve users' understanding of AI models, improve user performance and potentially reduce overreliance on AI predictions. However, this is mostly evaluated by static rather than dynamic measures, and the role of XAI on learning over trials is rarely studied. In this study, we use a context-free sequence prediction task, in which 458 participants predict the next symbol in a fixed sequence (with some noise) over 80 trials. We compare performance with AI and XAI advice against no AI support, and subsequently we test for learning by taking away the AI support after 40 trials (i.e., a reversal study design). Our results show that users learn faster with XAI than with AI without explanations or no AI and are better able to recover in performance from the removal of AI. However, the benefits of XAI on learning are much smaller for more difficult tasks. This work demonstrates the benefits of repeated measures user studies and multilevel modeling to better understand learning processes in XAI. It also shows the potential of AI explanations to help users to learn and poses XAI design suggestions to support learning in human-AI collaboration.