Unremarkable to Remarkable AI Agent: Exploring Boundaries of Agent Intervention for Adults With and Without Cognitive ImpairmentAs the population of older adults increases, there is a growing need for support for them to age in place. This is exacerbated by the growing number of individuals struggling with cognitive decline and shrinking number of youth who provide care for them. Artificially intelligent agents could provide cognitive support to older adults experiencing memory problems, and they could help informal caregivers with coordination tasks. To better understand this possible future, we conducted a speed dating with storyboards study to reveal invisible social boundaries that might keep older adults and their caregivers from accepting and using agents. We found that healthy older adults worry that accepting agents into their homes might increase their chances of developing dementia. At the same time, they want immediate access to agents that know them well if they should experience cognitive decline. Older adults in the early stages of cognitive decline expressed desire for agents that can ease the burden they saw themselves becoming for their caregivers. They also speculated that an agent who really knew them well might be an effective advocate for their needs when they were less able to advocate for themselves. That is, the agent may need to transition from being unremarkable to remarkable. Based on these findings, we present design opportunities and considerations for agents and articulate directions of future research.2025MCMai Lee Chang et al.Humanized AI: Avatars, Agents, and Voice AssistantsCSCW
CodeA11y: Making AI Coding Assistants Useful for Accessible Web DevelopmentA persistent challenge in accessible computing is ensuring developers produce web UI code that supports assistive technologies. Despite numerous specialized accessibility tools, novice developers often remain unaware of them, leading to ~96% of web pages that contain accessibility violations. AI coding assistants, such as GitHub Copilot, could offer potential by generating accessibility-compliant code, but their impact remains uncertain. Our formative study with 16 developers without accessibility training revealed three key issues in AI-assisted coding: failure to prompt AI for accessibility, omitting crucial manual steps like replacing placeholder attributes, and the inability to verify compliance. To address these issues, we developed CodeA11y, a GitHub Copilot Extension, that suggests accessibility-compliant code and displays manual validation reminders. We evaluated it through a controlled study with another 20 novice developers. Our findings demonstrate its effectiveness in guiding novice developers by reinforcing accessibility practices throughout interactions, representing a significant step towards integrating accessibility into AI coding assistants.2025PMPeya Mowar et al.Carnegie Mellon University, Robotics InstituteGenerative AI (Text, Image, Music, Video)Universal & Inclusive DesignCHI
Dynamic Agent Affiliation: Who Should the AI Agent Work for in the Older Adult's Care Network?The population of older adults experiencing cognitive decline is growing faster than the number of workers who can care for them. Artificially intelligent (AI) agents could assist these older adults, keeping them in their homes longer. For this to happen, older adults must be willing to adopt and rely on agents. Would they trust an agent that might need to report their decline to others? We conducted a speed dating study exploring the impact of agent affiliation (i.e., who the agent should work for). Our healthy and declining participants reacted positively to the idea of agents supporting them. They particularly recognized how the agent would reduce the burden placed on their family caregivers. They viewed affiliation to be dynamic, shifting from the declining older adult and orienting more to their caregivers over the course of cognitive decline. They envisioned the agent modifying its decision-making process to be like their caregivers'.2024MCMai Lee Chang et al.Elderly Care & Dementia SupportAging-in-Place Assistance SystemsHuman-Robot Collaboration (HRC)DIS
Recentering Reframing as an RtD Contribution: The Case of Pivoting from Accessible Web Tables to a Conversational Internet Design produces valuable knowledge by offering new perspectives that reframe problematic situations. Research through Design (RtD) contributes new frames along with design work demonstrating a frame’s value. Interestingly, RtD papers rarely describe how reframing happens. This gap in documentation unintentionally implies a romantic account of design, it implies that the first step of an RtD project is to have a brilliant idea. This is especially problematic in cases where the reframing causes a pivot that leads to a new research program. To help address this gap, we describe a case where through a series of three design experiments we experienced a research pivot. We describe how our work to improve web-table navigation for screen-reader users broke our frame. The break led to a new research program focused on constructing a conversational internet. This paper offers our case along with reflection on reporting design work that drives reframing.2022JZJohn Zimmerman et al.Carnegie Mellon UniversityVisual Impairment Technologies (Screen Readers, Tactile Graphics, Braille)Augmentative & Alternative Communication (AAC)Participatory DesignCHI
Social Robots in Service Contexts: Exploring the Rewards and Risks of Personalization and Re-embodimentSocial agents and robots are moving into front-line positions in brick and mortar services, taking on roles where they directly interact with customers. These agents could potentially recognize customers to personalize service. Will customers like this, or might they feel monitored and profiled? Robots could also re-embody (move their "personality" between one body and another) in order to take on multiple roles that are typically performed by different people. Will this make customers feel more taken care of, or will it raise concerns about the robot’s competence and expertise? Our work investigates when robots should and should not recognize customers and re-embody. Our online study used storyboards to present possible future interactions between robots and customers across several different service contexts. Our findings suggest that people generally accept robots identifying customers and taking on vastly different roles. However, in some contexts, these robot behaviors seem creepy and untrustworthy2021SRSamantha Reig et al.Agent Personality & AnthropomorphismSocial Robot InteractionDIS
Priorities, Technology & Power: Co-Designing an Inclusive Transit Agenda in Kampala, UgandaThere is considerable effort within the HCI community to explore, document, and advocate for the lived experiences of persons with disabilities (PWDs). However, PWDs from the Global South, particularly Africa, are underrepresented in this scholarship. We contribute to closing this gap by investigating the unmet transit needs and characterization of technology within the disability community in Kampala, Uganda. We investigated transportation due to the increase in ride-share solutions created by widespread mobile computing and the resulting disruption of transportation worldwide. We hosted co-design sessions with disability advocates and adapted the stakeholder tokens method from the value-sensitive design framework to map the stakeholder ecosystem. Our key insight is the identification of a new group of non-traditional core stakeholders who highlight the values of inclusion, mobility, and safety within the ecosystem. Finally, we discuss how our findings engage with concepts of disability justice and perceptions of power.2021LKLynn Kirabo et al.Carnegie Mellon UniversityRidesharing PlatformsParticipatory DesignCHI
Death of a Robot: Social Media Reactions and Language Usage in Response to Robot RetirementPeople take to social media to share their thoughts, joys, and sorrows. A recent popular trend has been to support and mourn people and pets that have died as well as other objects that have suffered catastrophic damage. As several popular robots have been discontinued, including the Opportunity Rover, Jibo, and Kuri, we are interested in how language used to mourn these robots compares to that to mourn people, animals, and other objects. We performed a study in which we asked participants to categorize deidentifed Twitter reactions as referencing the death of a person, an animal, a robot, or another object. Most reactions were labeled as being about humans, which suggests that people use similar language to describe feelings for animate and inanimate entities. We used a natural language toolkit to analyze language from a larger set of tweets. A majority of tweets about Opportunity included second-person (“you”) and gendered third-person pronouns (she/he versus it), but terms like “R.I.P” were reserved almost exclusively for humans and animals. Our findings suggest that people verbally mourn robots similarly to living things, but reserve some language for people.2020ECElizabeth Jeanne Carter et al.Social Platform Design & User BehaviorOnline Identity & Self-PresentationSocial Robot InteractionHRI
Not Some Random Agent: Multi-person interactions with a personalizing service robotService robots often perform their main functions in public settings, interacting with more than one person at a time. How these robots should handle the affairs of individual users while also behaving appropriately when others are present is an open question. One option is to design for flexible agent embodiment: letting agents take control of different robots as people move between contexts. Through structured User Enactments, we explored how agents embodied within a single robot might interact with multiple people. Participants interacted with a robot embodied by a singular service agent, agents that re-embody in different robots and devices, and agents that co-embody within the same robot. Findings reveal key insights about the promise of re-embodiment and co-embodiment as design paradigms as well as what people value during interactions with service robots that use personalization.2020SRSamantha Reig et al.Social Robot InteractionHuman-Robot Collaboration (HRC)HRI
Re-examining Whether, Why, and How Human-AI Interaction Is Uniquely Difficult to DesignArtificial Intelligence (AI) plays an increasingly important role in improving HCI and user experience. Yet many challenges persist in designing and innovating valuable human-AI interactions. For example, AI systems can make unpredictable errors, and these errors damage UX and even lead to undesired societal impact. However, HCI routinely grapples with complex technologies and mitigates their unintended consequences. What makes AI different? What makes human-AI interaction appear particularly difficult to design? This paper investigates these questions. We synthesize prior research, our own design and research experience, and our observations when teaching human-AI interaction. We identify two sources of AI's distinctive design challenges: 1) uncertainty surrounding AI's capabilities, 2) AI's output complexity, spanning from simple to adaptive complex. We identify four levels of AI systems. On each level, designers encounter a different subset of the design challenges. We demonstrate how these findings reveal new insights for designers, researchers, and design tool makers in productively addressing the challenges of human-AI interaction going forward.2020QYQian Yang et al.Carnegie Mellon UniversityGenerative AI (Text, Image, Music, Video)Explainable AI (XAI)AI-Assisted Decision-Making & AutomationCHI
Unremarkable AI: Fitting Intelligent Decision Support into Critical, Clinical Decision-Making ProcessesClinical decision support tools (DST) promise improved healthcare outcomes by offering data-driven insights. While effective in lab settings, almost all DSTs have failed in practice. Empirical research diagnosed poor contextual fit as the cause. This paper describes the design and field evaluation of a radically new form of DST. It automatically generates slides for clinicians' decision meetings with subtly embedded machine prognostics. This design took inspiration from the notion of Unremarkable Computing, that by augmenting the users' routines technology/AI can have significant importance for the users yet remain unobtrusive. Our field evaluation suggests clinicians are more likely to encounter and embrace such a DST. Drawing on their responses, we discuss the importance and intricacies of finding the right level of unremarkableness in DST design, and share lessons learned in prototyping critical AI systems as a situated experience.2019QYQian Yang et al.Carnegie Mellon UniversityAI-Assisted Decision-Making & AutomationCHI
Speak Up: A Multi-Year Deployment of Games to Motivate Speech Therapy in IndiaThe ability to communicate is crucial to leading an independent life. Unfortunately, individuals from developing communities who are deaf and hard of hearing tend to encounter difficulty communicating, due to a lack of educational resources. We present findings from a two-year deployment of Speak Up, a suite of voice-powered games to motivate speech therapy, at a school for the deaf in India. Using ethnographic methods, we investigated the interplay between Speak Up and local educational practices. We found that teachers' speech therapy goals had evolved to differ from those encoded in the games, that the games influenced classroom dynamics, and that teachers had improved their computer literacy and developed creative uses for the games. We used these insights to further enhance Speak Up by creating an explicit teacher role in the games, making changes that encouraged teachers to build their computer literacy, and adding an embodied agent.2018ANAmal Nanavati et al.Carnegie Mellon UniversityElectrical Muscle Stimulation (EMS)Shape-Changing Interfaces & Soft Robotic MaterialsSpecial Education TechnologyCHI
A Field Study Of Pedestrians And Autonomous VehiclesAutonomous vehicles have been in development for nearly thirty years and recently have begun to operate in real-world, uncontrolled settings. With such advances, more widespread research and evaluation of human interaction with autonomous vehicles (AV) is necessary. Here, we present an interview study of 32 pedestrians who have interacted with Uber AVs. Our findings are focused on understanding and trust of AVs, perceptions of AVs and artificial intelligence, and how the perception of a brand affects these constructs. We found an inherent relationship between favorable perceptions of technology and feelings of trust toward AVs. Trust in AVs was also influenced by a favorable interpretation of the company’s brand and facilitated by knowledge about what AV technology is and how it might fit into everyday life. To our knowledge, this paper is the first to surface AV-related interview data from pedestrians in a natural, real-world setting.2018SRSamantha Reig et al.External HMI (eHMI) — Communication with Pedestrians & CyclistsAutoUI