Understanding Interaction with ML through a Thematic Analysis Coding Assistant

Interactive Machine Learning (ML) enables users, including non-experts in ML, to iteratively train and improve ML models. However, limited research has been reported on how non-experts interact with these systems. Focusing on thematic analysis as a practical application, we report on a user study where 20 participants interacted with TACA, a functioning Interactive ML tool. Thematic analysis involves individual interpretation of ambiguous data, hence it is suited for and can benefit from the iterative customization of models supported by Interactive ML. Through a combination of interaction logs and semi-structured interviews, our findings revealed that, by using TACA, participants critically reflected on their analysis, gained new thematic insights, and adapted their interpretative stance. We also document misconceptions of ML concepts, positivist views, and personal blame for poor model performance. We then discuss how applications could be designed to improve the understanding of Interactive ML concepts and foster reflexive work practices.