TastePaths: Enabling Deeper Exploration and Understanding of Personal Preferences in Recommender SystemsRecommender systems are ubiquitous and influence the information we consume on a daily basis by helping us navigate vast catalogs of information like music databases. However, their linear approach of surfacing content in ranked lists limits their ability to help us grow and understand our personal preferences. In this paper, we study how we can better support users in exploring a novel space, specifically focusing on music genres. Informed by interviews with expert music listeners, we developed TastePaths: an interactive web-tool which helps users explore an overview of the genre-space via a graph of connected artists. We conducted a comparative user study with 16 participants where each of them used a personalized version of TastePaths (built with a set of artists the user listens to frequently) and a non-personalized one (based on a set of the most popular artists in the genre). We find that while participants employed various strategies to explore the space, overall they greatly preferred the personalized version as it helped anchor their exploration and provide recommendations that were more compatible with their personal taste. In addition to that, TastePaths helped participants specify and articulate their interest in the genre and gave them a better understanding of the system's organization of music. Based on our findings, we discuss opportunities and challenges for incorporating more control and expressive feedback in recommendation systems, in order to help users explore spaces beyond their immediate interests and improve these systems' underlying algorithms.2022SPSavvas Petridis et al.Recommender System UXInteractive Data VisualizationIUI
Where Responsible AI meets Reality: Practitioner Perspectives on Enablers for shifting Organizational PracticesLarge and ever-evolving technology companies continue to invest more time and resources to incorporate responsible Artificial Intelligence (AI) into production-ready systems to increase algorithmic accountability. This paper examines and seeks to offer a framework for analyzing how organizational culture and structure impact the effectiveness of responsible AI initiatives in practice. We present the results of semi-structured qualitative interviews with practitioners working in industry, investigating common challenges, ethical tensions, and effective enablers for responsible AI initiatives. Focusing on major companies developing or utilizing AI, we have mapped what organizational structures currently support or hinder responsible AI initiatives, what aspirational future processes and structures would best enable effective initiatives, and what key elements comprise the transition from current work practices to the aspirational future.2021BRBogdana Rakova et al.Algorithmic Auditing and Responsible AICSCW
Let Me Ask You This: How Can a Voice Assistant Elicit Explicit User Feedback?Voice assistants offer users access to an increasing variety of personalized functionalities. The researchers and engineers who build these experiences rely on various signals from users to create the machine learning models powering them. One type of signal is explicit in situ feedback. While collecting explicit in situ user feedback via voice assistants would help improve and inspect the underlying models, from a user perspective it can be disruptive to the overall experience, and the user might not feel compelled to respond. However, careful design can help alleviate friction in the experience. In this paper, we explore the opportunities and the design space for voice assistant feedback elicitation. First, we present four usage categories of explicit in-situ context for model evaluation and improvement, derived from interviews with machine learning practitioners. Then, using realistic scenarios generated for each category and based on examples from the interviews, we conducted an online study to evaluate multiple voice assistant designs. Our results reveal that when the voice assistant is framed as a learner or a collaborator, users were more willing to respond to its request for feedback and felt that the experience was less disruptive. In addition, giving users instructions on how to initiate feedback themselves can reduce the perceived disruptiveness to the experience compared to asking users for feedback directly in the form of a question. Based on our findings, we discuss the implications and potential future directions for designing voice assistants to elicit user feedback for personalized voice experiences.2021ZXZiang Xiao et al.Voice and SpeechCSCW
Towards Fairness in Practice: A Practitioner-Oriented Rubric for Evaluating Fair ML ToolkitsIn order to support fairness-forward thinking by machine learning (ML) practitioners, fairness researchers have created toolkits that aim to transform state-of-the-art research contributions into easily-accessible APIs. Despite these efforts, recent research indicates a disconnect between the needs of practitioners and the tools offered by fairness research. By engaging 20 ML practitioners in a simulated scenario in which they utilize fairness toolkits to make critical decisions, this work aims to utilize practitioner feedback to inform recommendations for the design and creation of fair ML toolkits. Through the use of survey and interview data, our results indicate that though fair ML toolkits are incredibly impactful on users’ decision-making, there is much to be desired in the design and demonstration of fairness results. To support the future development and evaluation of toolkits, this work offers a rubric that can be used to identify critical components of Fair ML toolkits.2021BRBrianna Richardson et al.SpotifyAI Ethics, Fairness & AccountabilityAlgorithmic Fairness & BiasCHI
Self-E: Smartphone-Supported Guidance for Customizable Self-ExperimentationThe ubiquity of self-tracking devices and smartphone apps has empowered people to collect data about themselves and try to self-improve. However, people with little to no personal analytics experience may not be able to analyze data or run experiments on their own (self-experiments). To lower the barrier to intervention-based self-experimentation, we developed an app called Self-E, which guides users through the experiment. We conducted a 2-week diary study with 16 participants from the local population and a second study with a more advanced group of users to investigate how they perceive and carry out self-experiments with the help of Self-E, and what challenges they face. We find that users are influenced by their preconceived notions of how healthy a given behavior is, making it difficult to follow Self-E's directions and trusting its results. We present suggestions to overcome this challenge, such as by incorporating empathy and scaffolding in the system.2021NDNediyana Daskalova et al.SpotifyMental Health Apps & Online Support CommunitiesFitness Tracking & Physical Activity MonitoringCHI
Using Remote Controlled Speech Agents to Explore Music Experience in ContextIt can be difficult for user researchers to explore how people might interact with interactive systems in everyday contexts; time and space limitations make it hard to be present everywhere that technology is used. Digital music services are one domain where designing for context is important given the myriad places people listen to music. One novel method to help design researchers embed themselves in everyday contexts is through remote-controlled speech agents. This paper describes a practitioner-centered case study of music service interaction researchers using a remote-controlled speech agent, called DJ Bot, to explore people’s music interaction in the car and the home. DJ Bot allowed the team to conduct remote user research and contextual inquiry and to quickly explore new interactions. However, challenges using a remote speech-agent arose when adapting DJ Bot from the constrained environment of the car to the unconstrained home environment.2020NMNikolas Martelaro et al.Voice User Interface (VUI) DesignIntelligent Voice Assistants (Alexa, Siri, etc.)Context-Aware ComputingDIS
Giving Voice to Silent Data: Designing with Personal Music Listening HistoryMusic streaming services collect listener data to support personalization and discovery of their extensive catalogs. Yet this data is typically used in ways that are not immediately apparent to listeners. We conducted design workshops with ten Spotify listeners to imagine future voice assistant (VA) interactions leveraging logged music data. We provided participants with detailed personal music listening data, such as play-counts and temporal patterns, which grounded their design ideas in their current behaviors. In the interactions participants designed, VAs did not simply speak their data out loud; instead, participants envisioned how data could implicitly support introspection, behavior change, and exploration. We present reflections on how VAs could evolve from voice-activated remote controls to intelligent music coaches and how personal data can be leveraged as a design resource.2020JWJordan Wirfs-Brock et al.University of Colorado, BoulderIntelligent Voice Assistants (Alexa, Siri, etc.)Music Composition & Sound Design ToolsCHI
Just Give Me What I Want: How People Use and Evaluate Music SearchMusic-streaming platforms offer users a large amount of content for consumption. Finding the right music can be challenging and users often need to search through extensive catalogs provided by these platforms. Prior research has focused on general-domain web search, which is designed to meet a broad range of user goals. Here, we study search in the domain of music, seeking to understand how and why people use search and how they evaluate their search experiences on a music-streaming platform. Over two studies, we conducted semi-structured interviews with 27 participants, asking about their search habits and preferences, and observing their behavior while searching for music. Analysis revealed participants evaluated their search experiences along two dimensions: success and effort. Importantly, how participants perceived success and effort differed by their mindset, or the way they assessed the results of their query. We conclude with recommendations to improve the user experience of music search.2019CHChristine Hosey et al.SpotifyRecommender System UXCHI
“Play PRBLMS”: Identifying and Correcting Less Accessible Content in Voice InterfacesVoice interfaces often struggle with specific types of named content. Domain-specific terminology and naming may push the bounds of standard language, especially in domains like music where artistic creativity extends beyond the music itself. Artists may name themselves with symbols (e.g. M△S▴C△RA) that most standard automatic speech recognition (ASR) systems cannot transcribe. Voice interfaces also experience difficulty surfacing content whose titles include non-standard spellings, symbols or other ASCII characters in place of English letters, or are written using a non-standard dialect. We present a generalizable method to detect content that current voice interfaces underserve by leveraging differences in engagement across input modalities. Using this detection method, we develop a typology of content types and linguistic practices that can make content hard to surface. Finally, we present a process using crowdsourced annotations to make underserved content more accessible.2018ASAaron Springer et al.Spotify, University of California Santa CruzVoice User Interface (VUI) DesignMultilingual & Cross-Cultural Voice InteractionVoice AccessibilityCHI
All Work and No Play? Conversations with a Question-and-Answer Chatbot in the WildMany conversational agents (CAs) are developed to answer users' questions in a specialized domain. In everyday use of CAs, user experience may extend beyond satisfying information needs to the enjoyment of conversations with CAs, some of which represent playful interactions. By studying a field deployment of a Human Resource chatbot, we report on users' interest areas in conversational interactions to inform the development of CAs. Through the lens of statistical modeling, we also highlight rich signals in conversational interactions for inferring user satisfaction with the instrumental usage and playful interactions with the agent. These signals can be utilized to develop agents that adapt functionality and interaction styles. By contrasting these signals, we shed light on the varying functions of conversational interactions. We discuss design implications for CAs, and directions for developing adaptive agents based on users' conversational behaviors.2018QLQ. Vera Liao et al.IBM T.J. Watson Research CenterConversational ChatbotsAgent Personality & AnthropomorphismCHI