Unremarkable to Remarkable AI Agent: Exploring Boundaries of Agent Intervention for Adults With and Without Cognitive ImpairmentAs the population of older adults increases, there is a growing need for support for them to age in place. This is exacerbated by the growing number of individuals struggling with cognitive decline and shrinking number of youth who provide care for them. Artificially intelligent agents could provide cognitive support to older adults experiencing memory problems, and they could help informal caregivers with coordination tasks. To better understand this possible future, we conducted a speed dating with storyboards study to reveal invisible social boundaries that might keep older adults and their caregivers from accepting and using agents. We found that healthy older adults worry that accepting agents into their homes might increase their chances of developing dementia. At the same time, they want immediate access to agents that know them well if they should experience cognitive decline. Older adults in the early stages of cognitive decline expressed desire for agents that can ease the burden they saw themselves becoming for their caregivers. They also speculated that an agent who really knew them well might be an effective advocate for their needs when they were less able to advocate for themselves. That is, the agent may need to transition from being unremarkable to remarkable. Based on these findings, we present design opportunities and considerations for agents and articulate directions of future research.2025MCMai Lee Chang et al.Humanized AI: Avatars, Agents, and Voice AssistantsCSCW
Exploring the Innovation Opportunities for Pre-trained ModelsInnovators transform the world by understanding where services are successfully meeting customers’ needs and then using this knowledge to identify failsafe opportunities for innovation. Pre-trained models have changed the AI innovation landscape, making it faster and easier to create new AI products and services. Understanding where pre-trained models are successful is critical for supporting AI innovation. Unfortunately, the hype cycle surrounding pre-trained models makes it hard to know where AI can really be successful. To address this, we investigated pre-trained model applications developed by HCI researchers as a proxy for commercially successful applications. The research applications demonstrate technical capabilities, address real user needs, and avoid ethical challenges. Using an artifact analysis approach, we categorized capabilities, opportunity domains, data types, and emerging interaction design patterns, uncovering some of the opportunity space for innovation with pre-trained models.2025MPMinjung Park et al.Generative AI (Text, Image, Music, Video)Human-LLM CollaborationExplainable AI (XAI)DIS
Making the Right Thing: Bridging HCI and Responsible AI in Early-Stage AI Concept SelectionAI projects often fail due to financial, technical, ethical, or user acceptance challenges—failures frequently rooted in early-stage decisions. While HCI and Responsible AI (RAI) research emphasize this, practical approaches for identifying promising concepts early remain limited. Drawing on Research through Design, this paper investigates how early-stage AI concept sorting in commercial settings can reflect RAI principles. Through three design experiments—including a probe study with industry practitioners—we explored methods for evaluating risks and benefits using multidisciplinary collaboration. Participants demonstrated strong receptivity to addressing RAI concerns early in the process and effectively identified low-risk, high-benefit AI concepts. Our findings highlight the potential of a design-led approach to embed ethical and service design thinking at the front end of AI innovation. By examining how practitioners reason about AI concepts, our study invites HCI and RAI communities to see early-stage innovation as a critical space for engaging ethical and commercial considerations together.2025JJJi-Youn Jung et al.AI Ethics, Fairness & AccountabilityParticipatory DesignSustainable HCIDIS
AI Mismatches: Identifying Potential Algorithmic Harms Before AI DevelopmentAI systems are often introduced with high expectations, yet many fail to deliver, resulting in unintended harm and missed opportunities for benefit. We frequently observe significant "AI Mismatches", where the system’s actual performance falls short of what is needed to ensure safety and co-create value. These mismatches are particularly difficult to address once development is underway, highlighting the need for early-stage intervention. Navigating complex, multi-dimensional risk factors that contribute to AI Mismatches is a persistent challenge. To address it, we propose an AI Mismatch approach to anticipate and mitigate risks early on, focusing on the gap between realistic model performance and required task performance. Through an analysis of 774 AI cases, we extracted a set of critical factors, which informed the development of seven matrices that map the relationships between these factors and highlight high-risk areas. Through case studies, we demonstrate how our approach can help reduce risks in AI development.2025DSDevansh Saxena et al.University of Wisconsin-Madison, The Information SchoolAI Ethics, Fairness & AccountabilityAlgorithmic Fairness & BiasCHI
Letters from Future Self: Augmenting the Letter-Exchange Exercise with LLM-based Agents to Enhance Young Adults' Career ExplorationYoung adults often encounter challenges in career exploration. Self-guided interventions, such as the letter-exchange exercise, where participants envision and adopt the perspective of their future selves by exchanging letters with their envisioned future selves, can support career development. However, the broader adoption of such interventions may be limited without structured guidance. To address this, we integrated Large Language Model (LLM)-based agents that simulate participants’ future selves into the letter-exchange exercise and evaluated their effectiveness. A one-week experiment (N=36) compared three conditions: (1) participants manually writing replies to themselves from the perspective of their future selves (baseline), (2) future-self agents generating letters to participants, and (3) future-self agents engaging in chat conversations with participants. Results indicated that exchanging letters with future-self agents enhanced participants' engagement during the exercise, while overall benefits of the intervention on future orientation, career self-concept, and psychological support remained comparable across conditions. We discuss design implications for AI-augmented interventions for supporting young adults' career exploration.2025HJHayeon Jeon et al.Seoul National University, Communication/Seoul National University/HCI+D LabHuman-LLM CollaborationAI-Assisted Creative WritingCHI
Exploring What People Need to Know to be AI Literate: Tailoring for a Diversity of AI Roles and ResponsibilitiesAI literacy research has had great success in offering competencies that capture the knowledge and skills users and developers of AI need to have for a world full of AI, helping them maximize its benefits and minimize its harms. However, recent years have witnessed other roles beyond users and developers whose responsibilities have been complicated by AI. In this work, we apply a service design approach to identify such roles and their responsibilities across various AI applications. By mapping the responsibilities to current AI literacy competencies, we exposed gaps suggesting unmet learning needs in current AI literacy research: identifying and assessing AI benefits, strategizing about AI’s benefits and risks, and monitoring and refining deployed AI to understand their changing impact. We discuss implications for future AI literacy research and its connection to Responsible AI research.2025SXShixian Xie et al.Carnegie Mellon University, Human-Computer Interaction InstituteExplainable AI (XAI)AI Ethics, Fairness & AccountabilityPrivacy Perception & Decision-MakingCHI
Integrating Equity in Public Sector Data-Driven Decision Making: Exploring the Desired Futures of Underserved StakeholdersPublic sectors aim to innovate not just for efficiency but also to enhance equity. Despite the growing adoption of data-driven decision-making systems in the public sector, efforts to integrate equity as a primary goal often fall short. This typically arises from inadequate early-stage involvement of the underserved stakeholders and prevalent misunderstandings concerning the authentic meaning of equity from these stakeholders' perspectives. Our research seeks to address this gap by actively involving undersevered stakeholders in the process of envisioning the integration of equity within public sector data-driven decisions, particularly in the context of a building department in a Northeastern mid-sized U.S. city. Applying a speed dating method with storyboards, we explore diverse equity-centric futures within the realm of local business development, a domain where small businesses, particularly women- and minority-owned businesses, historically confront inequitable distribution of public services. We explored three essential aspects of equity: monitoring equity, resource allocation prioritization, as well as information and equity. Our findings illuminate the complexities of integrating equity into data-driven decisions, offering nuanced insights about the needs of stakeholders. We found that attempts to monitor and incorporate equity goals into public sector decision-making can unexpectedly backfire, inadvertently sparking community apprehension and potentially exacerbating existing inequities. Small business owners, including those identifying as women- and minority-owned, advocated against the use of demographic-based data in equity-focused data-driven decision-making in the public sector, instead emphasizing factors like community needs, application complexity, and inherent small business uncertainties. Drawing from these insights, we propose design implications to assist designers of public sector data-driven decision-making systems better accommodate equity considerations.2024SKSeyun Kim et al.Session 2e: Data, Power, and JusticeCSCW
ClassID: Enabling Student Behavior Attribution from Ambient Classroom Sensing SystemsPatidar 等人开发ClassID环境课堂感知系统，通过多模态传感器和机器学习实现学生行为自动归因，帮助教师实时了解课堂动态与学生参与度。2024PPPrasoon Patidar et al.Intelligent Tutoring Systems & Learning AnalyticsCollaborative Learning & Peer TeachingUser Research Methods (Interviews, Surveys, Observation)UbiComp
"My Sense of Morality Leads to My Suffering, Battling, and Arguing": The Role of Platform Designers in (Un)Deciding Gig Worker IssuesHCI and design studies have increasingly identified challenges for gig workers and advocated for designs centered around worker justice. However, there's an existing research gap in understanding how platform designers approach gig worker issues in their practice. Our study engaged ten platform designers from food delivery and ride-hailing platforms to investigate this gap. Through semi-structured interviews, we uncovered their strategies, the extent of authority and responsibilities, and the range of obstacles they encounter in influencing decision-making that could affect gig workers’ experiences with the platforms. While platform designers were aware of gig worker issues, they confronted challenges from business goals, decision-making power, policies, and job security in promoting worker well-being. We discuss the jurisdiction of platform designers and propose that HCI research should further support them, who are deeply engaged in the gig economy and have the potential to participate in addressing social justice issues.2024SMShuhao Ma et al.Gig Economy PlatformsInclusive DesignDIS
Dynamic Agent Affiliation: Who Should the AI Agent Work for in the Older Adult's Care Network?The population of older adults experiencing cognitive decline is growing faster than the number of workers who can care for them. Artificially intelligent (AI) agents could assist these older adults, keeping them in their homes longer. For this to happen, older adults must be willing to adopt and rely on agents. Would they trust an agent that might need to report their decline to others? We conducted a speed dating study exploring the impact of agent affiliation (i.e., who the agent should work for). Our healthy and declining participants reacted positively to the idea of agents supporting them. They particularly recognized how the agent would reduce the burden placed on their family caregivers. They viewed affiliation to be dynamic, shifting from the declining older adult and orienting more to their caregivers over the course of cognitive decline. They envisioned the agent modifying its decision-making process to be like their caregivers'.2024MCMai Lee Chang et al.Elderly Care & Dementia SupportAging-in-Place Assistance SystemsHuman-Robot Collaboration (HRC)DIS
Investigating Why Clinicians Deviate from Standards of Care: Liberating Patients from Mechanical Ventilation in the ICUClinical practice guidelines, care pathways, and protocols are designed to support evidence-based practices for clinicians; however, their adoption remains a challenge. We set out to investigate why clinicians deviate from the "Wake Up and Breathe" protocol, an evidence-based guideline for liberating patients from mechanical ventilation in the intensive care unit (ICU). We conducted over 40 hours of direct observations of live clinical workflows, 17 interviews with frontline care providers, and 4 co-design workshops at three different medical intensive care units. Our findings indicate that unlike prior literature suggests, disagreement with the protocol is not a substantial barrier to adoption. Instead, the uncertainty surrounding the application of the protocol for individual patients leads clinicians to deprioritize adoption in favor of tasks where they have high certainty. Reflecting on these insights, we identify opportunities for technical systems to help clinicians in effectively executing the protocol and discuss future directions for HCI research to support the integration of protocols into clinical practice in complex, team-based healthcare settings.2024NYNur Yildirim et al.Carnegie Mellon UniversityMental Health Apps & Online Support CommunitiesUser Research Methods (Interviews, Surveys, Observation)CHI
The Future of HCI-Policy CollaborationPolicies significantly shape computation's societal impact, a crucial HCI concern. However, challenges persist when HCI professionals attempt to integrate policy into their work or affect policy outcomes. Prior research considered these challenges at the "border" of HCI and policy. This paper asks: What if HCI considers policy integral to its intellectual concerns, placing system-people-policy interaction not at the border but nearer the center of HCI research, practice, and education? What if HCI fosters a mosaic of methods and knowledge contributions that blend system, human, and policy expertise in various ways, just like HCI has done with blending system and human expertise? We present this re-imagined HCI-policy relationship as a provocation and highlight its usefulness: It spotlights previously overlooked system-people-policy interaction work in HCI. It unveils new opportunities for HCI's futuring, empirical, and design projects. It allows HCI to coordinate its diverse policy engagements, enhancing its collective impact on policy outcomes.2024QYMing Yang et al.Cornell UniversityAlgorithmic Fairness & BiasInclusive DesignTechnology Ethics & Critical HCICHI
Sketching AI Concepts with Capabilities and Examples: AI Innovation in the Intensive Care UnitAdvances in artificial intelligence (AI) have enabled unprecedented capabilities, yet innovation teams struggle when envisioning AI concepts. Data science teams think of innovations users do not want, while domain experts think of innovations that cannot be built. A lack of effective ideation seems to be a breakdown point. How might multidisciplinary teams identify buildable and desirable use cases? This paper presents a first hand account of ideating AI concepts to improve critical care medicine. As a team of data scientists, clinicians, and HCI researchers, we conducted a series of design workshops to explore more effective approaches to AI concept ideation and problem formulation. We detail our process, the challenges we encountered, and practices and artifacts that proved effective. We discuss the research implications for improved collaboration and stakeholder engagement, and discuss the role HCI might play in reducing the high failure rate experienced in AI innovation.2024NYNur Yildirim et al.Carnegie Mellon UniversityGenerative AI (Text, Image, Music, Video)Human-LLM CollaborationMental Health Apps & Online Support CommunitiesCHI
ClassInSight: Designing Conversation Support Tools to Visualize Classroom Discussion for Personalized Teacher Professional DevelopmentTeaching is one of many professions for which personalized feedback and reflection can help improve dialogue and discussion between the professional and those they serve. However, professional development (PD) is often impersonal as human observation is labor-intensive. Data-driven PD tools in teaching are of growing interest, but open questions about how professionals engage with their data in practice remain. In this paper, we present ClassInSight, a tool that visualizes three levels of teachers’ discussion data and structures reflection. Through 22 reflection sessions and interviews with 5 high school science teachers, we found themes related to dissonance, contextualization, and sustainability in how teachers engaged with their data in the tool and in how their professional vision, the use of professional expertise to interpret events, shifted over time. We discuss guidelines for these conversational support tools to support personalized PD in professions beyond teaching where conversation and interaction are important.2024TNTricia J. Ngoon et al.Carnegie Mellon UniversityInteractive Data VisualizationPrototyping & User TestingCHI
"An Instructor is [already] able to keep track of 30 students": Students’ Perceptions of Smart Classrooms for Improving Teaching & Their Emergent Understandings of Teaching and LearningMulti-modal classroom sensing systems can collect complex behaviors in the classroom at a scale and precision far greater than human observers to capture learning insights and provide personalized teaching feedback. As students are critical stakeholders in the adoption of smart classrooms for the improvement of teaching, open questions remain in understanding student perspectives on the use of their data to provide insights to instructors. We conducted a Speed Dating with storyboards study to explore student values and boundaries regarding the acceptance of classroom sensing systems in STEM college courses. We found that students have several emergent beliefs about teaching and learning that influence their views towards smart classroom technologies. Students also held contextual views on the boundaries of data use depending on the outcome. Our findings have implications for the design and communication of classroom sensing systems that reconcile student and instructor beliefs around teaching and learning.2023TNTricia J. Ngoon et al.Intelligent Tutoring Systems & Learning AnalyticsSTEM Education & Science CommunicationDIS
Creating Design Resources to Scaffold the Ideation of AI ConceptsAdvances in artificial intelligence have enabled unprecedented technical capabilities, yet making these advances useful in the real world remains challenging. We engaged in a Research through Design process to improve the ideation of AI products and services. We developed a design resource capturing AI capabilities based on 40 AI features commonly used across various domains. To probe its usefulness, we created a set of slides illustrating AI capabilities and asked designers to ideate AI-enabled user experiences. We also incorporated capabilities into our own design process to brainstorm concepts with domain experts and data scientists. Our research revealed that designers should focus on innovations where moderate AI performance creates value. We reflect on our process and discuss research implications for creating and assessing resources to systematically explore AI’s problem-solution space.2023NYNur Yildirim et al.Generative AI (Text, Image, Music, Video)Human-LLM CollaborationPrototyping & User TestingDIS
Uncovering Gig Worker-Centered Design Opportunities in Food Delivery WorkThe gig economy and digital labor platforms, such as food delivery, have become essential while also troubling the current socioeconomic landscape. Delivery platforms promise entry-level work, flexibility, and other benefits. However, researchers remain divided on if these platforms benefit workers and society at large. This study aims to shed light on the comprehensive challenges in food delivery work, uncovering gig worker-centered design opportunities to improve the lives of food couriers. Adopting an exploratory research process, we analyzed 19 ride-along food delivery videos and performed nine semi-structured interviews with food couriers in Portugal. Our findings illustrated the complexity and challenging nature of delivery work due to the entangled physical, digital, social, natural, and human factors. We captured and discussed gig worker-centered opportunities that surfaced from work challenges, echoing the needs of food couriers about supporting work, justice, inclusion, and work vision.2023SMShuhao Ma et al.Elderly Care & Dementia SupportGig Economy PlatformsParticipatory DesignDIS
Recentering Reframing as an RtD Contribution: The Case of Pivoting from Accessible Web Tables to a Conversational Internet Design produces valuable knowledge by offering new perspectives that reframe problematic situations. Research through Design (RtD) contributes new frames along with design work demonstrating a frame’s value. Interestingly, RtD papers rarely describe how reframing happens. This gap in documentation unintentionally implies a romantic account of design, it implies that the first step of an RtD project is to have a brilliant idea. This is especially problematic in cases where the reframing causes a pivot that leads to a new research program. To help address this gap, we describe a case where through a series of three design experiments we experienced a research pivot. We describe how our work to improve web-table navigation for screen-reader users broke our frame. The break led to a new research program focused on constructing a conversational internet. This paper offers our case along with reflection on reporting design work that drives reframing.2022JZJohn Zimmerman et al.Carnegie Mellon UniversityVisual Impairment Technologies (Screen Readers, Tactile Graphics, Braille)Augmentative & Alternative Communication (AAC)Participatory DesignCHI
How Experienced Designers of Enterprise Applications Engage AI as a Design MaterialHCI research has explored AI as a design material, suggesting that designers can envision AI's design opportunities to improve UX. Recent research claimed that enterprise applications offer an opportunity for AI innovation at the user experience level. We conducted design workshops to explore the practices of experienced designers who work on cross-functional AI teams in the enterprise. We discussed how designers successfully work with and struggle with AI. Our findings revealed that designers can innovate at the system and service levels. We also discovered that making a case for an AI feature's return on investment is a barrier for designers when they propose AI concepts and ideas. Our discussions produced novel insights on designers' role on AI teams, and the boundary objects they used for collaborating with data scientists. We discuss the implications of these findings as opportunities for future research aiming to empower designers in working with data and AI.2022NYNur Yildirim et al.Carnegie Mellon UniversityGenerative AI (Text, Image, Music, Video)AI-Assisted Decision-Making & AutomationCHI
Social Robots in Service Contexts: Exploring the Rewards and Risks of Personalization and Re-embodimentSocial agents and robots are moving into front-line positions in brick and mortar services, taking on roles where they directly interact with customers. These agents could potentially recognize customers to personalize service. Will customers like this, or might they feel monitored and profiled? Robots could also re-embody (move their "personality" between one body and another) in order to take on multiple roles that are typically performed by different people. Will this make customers feel more taken care of, or will it raise concerns about the robot’s competence and expertise? Our work investigates when robots should and should not recognize customers and re-embody. Our online study used storyboards to present possible future interactions between robots and customers across several different service contexts. Our findings suggest that people generally accept robots identifying customers and taking on vastly different roles. However, in some contexts, these robot behaviors seem creepy and untrustworthy2021SRSamantha Reig et al.Agent Personality & AnthropomorphismSocial Robot InteractionDIS