From Bias to Repair: Error as a Site of Collaboration and Negotiation in Applied Data Science Work

The management of error has become an increasingly central and contested question in data science work. While much recent scholarship in artificial intelligence and machine learning has focused on limiting and eliminating error, practitioners have also long used error as a site of collaboration and learning vis-à-vis labelers, domain experts and the specific worlds data scientists seek to model and understand. Drawing on work in CSCW, STS, HCML and repair studies, as well as multisited ethnographic fieldwork in a government institution and a non-profit organization, we move beyond the notion of error as edge case or anomaly to make three basic arguments: first, that error discloses or calls to attention existing structures of collaboration unseen or underappreciated under ‘working’ systems; second, that error calls into being new forms and sites of collaboration (including, sometimes, new actors); and third, that error redeploys old sites and actors in new ways, including by restructuring relations of hierarchy and expertise that may alternately recenter and/or devalue the position of different actors. We conclude by discussing how an artful living with error can better support creative strategies of negotiation and adjustment that data scientists and their collaborators engage in when faced with disruption, breakdown, and frictions in AI.