How do People Train a Machine? Strategies and (Mis)Understandings

Video

DOI

Machine learning has become pervasive in modern interactive technology due to the wide range of complex tasks it can handle. However, most machine learning systems provide users with surprisingly little, if any, agency with respect to how their models are trained from data. In this paper, we explore the way people could handle learning algorithms, what they understand from their behavior and what strategy they may use to “make it work”. To address these questions, we developed an web-based sketch-based recognition algorithm, called Marcelle-Sketch, that end-users can teach. We present two experimental studies that investigate people's strategies, beliefs and (mis)understandings in a realistic algorithm-teaching task. Study one took place in online workshop that collected drawing data from 22 novice users and analyzed their teaching strategies. Study two involved eight participants who performed a similar task during individual teaching sessions, using a think-aloud protocol. Our results show that users have different inputs scheduling. Their strategy incorporate investigations of the model's capabilities using input variability, driving changes in users understanding of machine learning during the session.We conclude with implications for the design of richer, more human-centered forms of interactions with machine learning, and impact for ML education and democratization.