Davis Rule

Interpretability Gone Bad: The Role of Bounded Rationality in How Practitioners Understand Machine Learning

While interpretability tools are intended to help people better understand machine learning (ML), we find that they can, in fact, impair understanding. This paper presents a pre-registered, controlled experiment showing that ML practitioners (N=119) spent 5x less time on task, and were 17% less accurate about the data…