Sustaining Workers Who Sustain the World: Assets-Based Design for Conservation Technologies in MadagascarLocal workers and their knowledge are essential for sustainable and effective conservation efforts. However, many technology-assisted conservation programs are guided by global benchmarks (e.g., forest cover) and industry metrics (e.g., cost per acre), which often devalue local knowledge and fail to consider the economic and conservation goals of local workers. Assets-based design is well-suited to center workers and their strengths, yet it may fail to fully address the complexities of long-term conservation programs by not explicitly emphasizing workers’ goals or bolstering their assets. We extend recent approaches in assets-based design literature that address these limitations through our case studies of reforestation, biodiversity monitoring, and carbon sequestration programs in three protected areas in Madagascar. We leverage a mixed-methods approach of direct reactive observations, unstructured interviews, and an informal design workshop, revealing emergent themes surrounding economic sustainability and the value of local ecological knowledge in conservation. Finally, we explore examples, tensions, and design considerations for worker-centered conservation technology to: (1) prioritize local knowledge, (2) foster love of nature, (3) center economic goals, and (4) embrace local autonomy. This work advances the dialogue on assets-based design, promoting the co-creation of equitable and sustainable conservation technologies with workers in Global South settings by centering local economic priorities and enhancing workers' strengths.2025EGEric Greenlee et al.Social and Environmental JusticeCSCW
POET: Supporting Prompting Creativity and Personalization with Automated Expansion of Text-to-Image GenerationState-of-the-art visual generative AI tools hold immense potential to assist users in the early ideation stages of creative tasks -- offering the ability to generate (rather than search for) novel and unprecedented (instead of existing) images of considerable quality that also adhere to boundless combinations of user specifications. However, many large-scale text-to-image systems are designed for broad applicability, yielding conventional output that may limit creative exploration. They also employ interaction methods that may be difficult for beginners. Given that creative end users often operate in diverse, context-specific ways that are often unpredictable, more variation and personalization are necessary. We introduce POET, a real-time interactive tool that (1) automatically discovers dimensions of homogeneity in text-to-image generative models, (2) expands these dimensions to diversify the output space of generated images, and (3) learns from user feedback to personalize expansions. An evaluation with 28 users spanning four creative task domains demonstrated POET's ability to generate results with higher perceived diversity and help users reach satisfaction in fewer prompts during creative tasks, thereby prompting them to deliberate and reflect more on a wider range of possible produced results during the co-creative process. Focusing on visual creativity, POET offers a first glimpse of how interaction techniques of future text-to-image generation tools may support and align with more pluralistic values and the needs of end users during the ideation stages of their work.2025EHMehtab Khan et al.Generative AI (Text, Image, Music, Video)AI-Assisted Creative WritingUIST
Toward Patient-Centered AI Fact Labels: Leveraging Extrinsic Trust CuesAI technologies in healthcare hold great promise for addressing numerous challenges, but ensuring that patients understand, trust, and adopt these technologies remains a significant hurdle. While the HCI community has proposed AI documentation frameworks (e.g., model cards) to enhance understanding, patient perspectives in the healthcare AI documentation remain underexplored. To address this gap, we designed prototypes based on existing frameworks and gathered feedback from 18 participants to explore their perspectives on AI documentation in cardiology, a domain where high-stakes AI tools are increasingly used and understanding users' trust in AI is essential. Our findings revealed patient needs for more detailed information about healthcare AI technologies, the importance of extrinsic trust cues (e.g., regulatory status), and the integration of AI documentation into existing care processes. Based on these findings, we discuss two design implications: enhancing patient-centeredness in AI documentation and leveraging extrinsic trust cues to improve its design. This study contributes to the HCI community by amplifying the patient voice in designing AI documentation and offering actionable insights into leveraging extrinsic trust cues effectively.2025DYDong Whi Yoo et al.Explainable AI (XAI)AI-Assisted Decision-Making & AutomationAI Ethics, Fairness & AccountabilityDIS
Can AI Model the Complexities of Human Moral Decision-making? A Qualitative Study of Kidney Allocation DecisionsA growing body of work in Ethical AI attempts to capture human moral judgments through simple computational models. The key question we address in this work is whether such simple AI models capture the critical nuances of moral decision-making by focusing on the use case of kidney allocation. We conducted twenty interviews where participants explained their rationale for their judgments about who should receive a kidney. We observe participants: (a) value patients' morally-relevant attributes to different degrees; (b) use diverse decision-making processes, citing heuristics to reduce decision complexity; (c) can change their opinions; (d) sometimes lack confidence in their decisions (e.g., due to incomplete information); and (e) express enthusiasm and concern regarding AI assisting humans in kidney allocation decisions. Based on these findings, we discuss challenges of computationally modeling moral judgments as a stand-in for human input, highlight drawbacks of current approaches, and suggest future directions to address these issues.2025VKVijay Keswani et al.Duke UniversityExplainable AI (XAI)AI Ethics, Fairness & AccountabilityPrivacy Perception & Decision-MakingCHI
RemiHaven: Integrating "In-Town" and "Out-of-Town" Peers to Provide Personalized Reminiscence Support for Older DriftersWith increasing social mobility and an aging society, more older adults in China are migrating to new cities, known as “older drifters”. Due to fewer social connections and cultural adaptation, they face negative emotions such as loneliness and depression. While reminiscence-based interventions have been used to improve older adults' psychological well-being, challenges such as the lack of tangible materials and limited social resources constrain the feasibility of traditional reminiscence approaches for older drifters. To address this challenge, we designed RemiHaven, a personalized reminiscence support tool based on a two-phase formative study. It integrates “In-Town” and “Out-of-Town” peer agents to enhance personalization, engagement, and emotional resonance in the reminiscence process powered by Multimodal Large Language Models (MLLMs). Our evaluations show RemiHaven's strengths in supporting reminiscence while identifying potential challenges. We conclude by offering insights for the future design of reminiscence support tools for older migrants.2025XZXuechen Zhang et al.Fudan UniversityHuman-LLM CollaborationMental Health Apps & Online Support CommunitiesElderly Care & Dementia SupportCHI
Realism Drives Interpersonal Reciprocity but Yields to AI-Assisted Egocentrism in a Coordination ExperimentVirtual reality technologies that enhance realism and artificial intelligence (AI) systems that assist human behavior are increasingly interwoven in social applications. However, how these technologies might jointly influence interpersonal coordination remains unclear. We conducted an experiment with 240 participants in 120 pairs who interacted through remote-controlled robot cars in a physical space or virtual cars in a digital space, with or without autosteering assistance, using the chicken game, an established model of interpersonal coordination. We find that both realism and AI assistance help improve user performance but through opposing mechanisms. Real-world contexts enhanced communication, fostering reciprocal actions and collective benefits. In contrast, autosteering assistance diminished the need for interpersonal coordination, shifting participants’ focus towards self-interest. Notably, when combined, the egocentric effects of autosteering assistance outweighed the prosocial effects of realism. The design of HCI systems that involve social coordination will, we believe, need to take such effects into account.2025HSHirokazu Shirado et al.Carnegie Mellon University, School of Computer ScienceAutomated Driving Interface & Takeover DesignHuman-Robot Collaboration (HRC)Technology Ethics & Critical HCICHI
BallistoBud: Heart Rate Variability Monitoring using Earbud Accelerometry for Stress AssessmentThis paper examines the potential of commercial earbuds for detecting physiological biomarkers like heart rate (HR) and heart rate variability (HRV) for stress assessment. Using accelerometer (IMU) and photoplethysmography (PPG) data from earbuds, we compared these estimates with reference electrocardiogram (ECG) data from 81 healthy participants. We explored using low-power accelerometer sensors for capturing ballistocardiography (BCG) signals. However, BCG signal quality can vary due to individual differences and body motion. Therefore, BCG data quality assessment is critical before extracting any meaningful biomarkers. To address this, we introduced the ECG-gated BCG heatmap, a new method for assessing BCG signal quality. We trained a Random Forest model to identify usable signals, achieving 82% test accuracy. Filtering out unusable signals improved HR/HRV estimation accuracy to levels comparable to PPG-based estimates. Our findings demonstrate the feasibility of accurate physiological monitoring with earbuds, advancing the development of user-friendly wearable health technologies for stress management.2025MIMd Saiful Islam et al.University of Rochester, Department of Computer Science; Samsung Research America, Digital HealthSleep & Stress MonitoringBiosensors & Physiological MonitoringCHI
Neural Canvas: Supporting Scenic Design Prototyping by Integrating 3D Sketching and Generative AIWe propose Neural Canvas, a lightweight 3D platform that integrates sketching and a collection of generative AI models to facilitate scenic design prototyping. Compared with traditional 3D tools, sketching in a 3D environment helps designers quickly express spatial ideas, but it does not facilitate the rapid prototyping of scene appearance or atmosphere. Neural Canvas integrates generative AI models into a 3D sketching interface and incorporates four types of projection operations to facilitate 2D-to-3D content creation. Our user study shows that Neural Canvas is an effective creativity support tool, enabling users to rapidly explore visual ideas and iterate 3D scenic designs. It also expedites the creative process for both novices and artists who wish to leverage generative AI technology, resulting in attractive and detailed 3D designs created more efficiently than using traditional modeling tools or individual generative AI platforms.2024YSYulin Shen et al.The Hong Kong University of Science and Technology (Guangzhou)Generative AI (Text, Image, Music, Video)3D Modeling & AnimationCreative Collaboration & Feedback SystemsCHI
Human-Algorithmic Interaction Using a Large Language Model-Augmented Artificial Intelligence Clinical Decision Support SystemIntegration of artificial intelligence (AI) into clinical decision support systems (CDSS) poses a socio-technological challenge that is impacted by usability, trust, and human-computer interaction (HCI). AI-CDSS interventions have shown limited benefit in clinical outcomes, which may be due to insufficient understanding of how health-care providers interact with AI systems. Large language models (LLMs) have the potential to enhance AI-CDSS, but haven't been studied in either simulated or real-world clinical scenarios. We present findings from a randomized controlled trial deploying AI-CDSS for the management of upper gastrointestinal bleeding (UGIB) with and without an LLM interface within realistic clinical simulations for physician and medical student participants. We find evidence that LLM augmentation improves ease-of-use, that LLM-generated responses with citations improve trust, and HCI varies based on clinical expertise. Qualitative themes from interviews suggest the perception of LLM-augmented AI-CDSS as a team-member used to confirm initial clinical intuitions and help evaluate borderline decisions.2024NRNiroop Channa Rajashekar et al.Yale School of MedicineHuman-LLM CollaborationExplainable AI (XAI)AI-Assisted Decision-Making & AutomationCHI
Alchemist: LLM-Aided End-User Development of Robot ApplicationsLarge Language Models (LLMs) have the potential to catalyze a paradigm shift in end-user robot programming-moving from the conventional process of user specifying programming logic to an iterative, collaborative process in which the user specifies desired program outcomes while LLM produces detailed specifications.We introduce a novel integrated development system, Alchemist, that leverages LLMs to empower end-users in creating, testing, and running robot programs using natural language inputs, aiming to reduce the required knowledge for developing robot applications.We present a detailed examination of our system design and provide an exploratory study involving true end-users to assess capabilities, usability, and limitations of our system.Through the design, development, and evaluation of our system, we derive a set of lessons learned from the use of LLMs in robot programming.We discuss how LLMs may be the next frontier for democratizing end-user development of robot applications.2024UKUlas Berk Karli et al.Generative AI (Text, Image, Music, Video)Human-LLM CollaborationHuman-Robot Collaboration (HRC)HRI
Designing Voice-First Ambient Interfaces to Support Aging in PlaceWe focus on the stories of five older adults who became voice assistant users through our study, and with whom we speculated about future interfaces through two design probes, one for health data reporting and one for positive reminiscing. We delivered a voice-first ambient interface (VFAI) to each participant, and closely observed participants' journeys through periodic themed interviews (16 hours, 21 minutes of transcribed recordings), usage log reviews (4,657 entries), and phone and text support. Participants' lived experiences impacted their perceptions and interactions with their VFAI, fueling rich insights about how to design for diverse needs. For example, while one participant saw increased potential in the VFAI after interacting with the design probe for health data reporting, another was skeptical of using it to communicate with her doctor. We contribute an in-depth exploration of VFAIs to support aging in place, implications for design, and areas for future work for tailoring VFAIs towards enabling continuity of care in people's homes.2023ACAndrea Cuadra et al.Aging-Friendly Technology DesignHome Voice Assistant ExperienceAging-in-Place Assistance SystemsDIS
Lively: Enabling Multimodal, Lifelike, and Extensible Real-time Robot MotionRobots designed to interact with people in collaborative or social scenarios must move in ways that are consistent with the robot's task and communication goals. However, combining these goals in a naïve manner can result in mutually exclusive solutions, or infeasible or problematic states and actions. In this paper, we present Lively, a framework which supports configurable, real-time, task-based and communicative or socially-expressive motion for collaborative and social robotics across multiple levels of programmatic accessibility. Lively supports a wide range of control methods (i.e. position, orientation, and joint-space goals), and balances them with complex procedural behaviors for natural, lifelike motion that are effective in collaborative and social contexts. We discuss the design of three levels of programmatic accessibility of Lively, including a graphical user interface for visual design called LivelyStudio, the core library Lively for full access to its capabilities for developers, and an extensible architecture for greater customizability and capability.2023ASAndrew Schoen et al.Social Robot InteractionHuman-Robot Collaboration (HRC)HRI
Interactive Policy Shaping for Human-Robot Collaboration with Transparent Matrix OverlaysOne important aspect of effective human--robot collaborations is the ability for robots to adapt quickly to the needs of humans. While techniques like deep reinforcement learning have demonstrated success as sophisticated tools for learning robot policies, the fluency of human-robot collaborations is often limited by these policies' inability to integrate changes to a user's preferences for the task. To address these shortcomings, we propose a novel approach that can modify learned policies at execution time via symbolic if-this-then-that rules corresponding to a modular and superimposable set of low-level constraints on the robot's policy. These rules, which we call Transparent Matrix Overlays, function not only as succinct and explainable descriptions of the robot’s current strategy but also as an interface by which a human collaborator can easily alter a robot's policy via verbal commands. We demonstrate the efficacy of this approach on a series of proof-of-concept cooking tasks performed in simulation and on a physical robot.2023JBJake Brawer et al.Explainable AI (XAI)Human-Robot Collaboration (HRC)HRI
Verbally Soliciting Human Feedback in Continuous Human-Robot Collaboration: Effects of the Framing and Timing of RemindersHumans expect robots to learn from their feedback and adapt to their preferences. However, there are limitations with how humans provide feedback to robots, e.g., humans may give less feedback as interactions progress. Therefore, it would be advantageous if robots could influence humans to provide more feedback during interactions. We conducted a 2x2 between-subjects user study (N=71) to investigate whether the framing and timing of a robot's reminder to provide feedback could influence human interactants. Human-robot interactions took place in the context of Space Invaders, a fast-paced and continuous collaborative environment. Our results suggest that reminders can influence the amount of feedback humans provide to robots, how participants feel about the robot, and how they feel about providing feedback during the interaction.2023KCKate Candon et al.Human-Robot Collaboration (HRC)HRI
Self-Annotation Methods for Aligning Implicit and Explicit Human Feedback in Human-Robot InteractionRecent research in robot learning suggests that implicit human feedback is a low-cost approach to improving robot behavior without the typical teaching burden on users. Because implicit feedback can be difficult to interpret, though, we study different methods to collect fine-grained labels from users about robot performance across multiple dimensions, which can then serve to map implicit human feedback to performance values. In particular, we focused on understanding the effects of annotation order and frequency on human perceptions of the self-annotation process and the usefulness of the labels for creating data-driven models to reason about implicit feedback. Our results demonstrate that different annotation methods can influence perceived memory burden, annotation difficulty, and overall annotation time. Based on our findings, we conclude with recommendations to create future implicit feedback datasets in Human-Robot Interaction.2023QZQiping Zhang et al.User Research Methods (Interviews, Surveys, Observation)Prototyping & User TestingHRI
Online Harassment in Majority Contexts: Examining Harms and Remedies across CountriesOnline harassment is a global problem. This article examines perceptions of harm and preferences for remedies associated with online harassment with nearly 4000 participants in 14 countries around the world. The countries in this work reflect a range of identities and values, with a focus on those outside of North American and European contexts. Results show that perceptions of harm are higher among participants from all countries studied compared to the United States. Non-consensual sharing of sexual photos is consistently rated as harmful in all countries, while insults and rumors are perceived as more harmful in non-U.S. countries, especially harm to family reputation. Lower trust in other people and lower trust in sense of safety in one's neighborhood correlate with increased perceptions of harm of online harassment. In terms of remedies, participants in most countries prefer monetary compensation, apologies, and publicly revealing offender's identities compared to the U.S. Social media platform design and policy must consider regional values and norms, which may depart from U.S. centric-approaches.2023SSSarita Schoenebeck et al.University of MichiganOnline Harassment & Counter-ToolsMisinformation & Fact-CheckingCHI
PointShopAR: Supporting Environmental Design Prototyping Using Point Cloud in Augmented RealityWe present PointShopAR, a novel tablet-based system for AR environmental design using point clouds as the underlying representation. It integrates point cloud capture and editing in a single AR workflow to help users quickly prototype design ideas in their spatial context. We hypothesize that point clouds are well suited for prototyping, as they can be captured more rapidly than textured meshes and then edited immediately in situ on the capturing device. We based the design of PointShopAR on the practical needs of six architects in a formative study. Our system supports a variety of point cloud editing operations in AR, including selection, transformation, hole filling, drawing, morphing, and animation. We evaluate PointShopAR through a remote study on usability and an in-person study on environmental design support. Participants were able to iterate design rapidly, showing the merits of an integrated capture and editing workflow with point clouds in AR environmental design.2023ZWZeyu Wang et al.The Hong Kong University of Science and Technology (Guangzhou), The Hong Kong University of Science and TechnologyAR Navigation & Context AwarenessPrototyping & User TestingCHI
Revisiting Group Fairness Metrics: The Effect of NetworksThere is an increasing amount of work studying fairness in socio-technical settings from a computational perspective. In this context, a variety of metrics have been introduced to measure fairness in different settings. Most of these metrics, however, do not account for the interactions between individuals or evaluate any underlying network's effect on the outcomes measured. While a wide body of work studies the organization of individuals into a network structure and how individuals access resources in networks, the impact of network structure on fairness has been largely unexplored. We introduce templates for group fairness metrics that account for network structure. More specifically, we present two types of group fairness metrics that measure distinct yet complementary forms of bias in networks. The first type of metric evaluates how access to others in the network is distributed across groups. The second type of metric evaluates how groups distribute their interactions across other groups, and hence captures inter-group biases. We find that ignoring the network can lead to spurious fairness evaluations by either not capturing imbalances in influence and reach illuminated by the first type of metric, or by overlooking interaction biases as evaluated by the second type of metric. Our empirical study illustrates these pronounced differences between network and non-network evaluations of fairness.2022AMAnay Mehrotra et al.Data, Bias and FairnessCSCW
Revisiting Group Fairness Metrics: The Effect of NetworksThere is an increasing amount of work studying fairness in socio-technical settings from a computational perspective. In this context, a variety of metrics have been introduced to measure fairness in different settings. Most of these metrics, however, do not account for the interactions between individuals or evaluate any underlying network's effect on the outcomes measured. While a wide body of work studies the organization of individuals into a network structure and how individuals access resources in networks, the impact of network structure on fairness has been largely unexplored. We introduce templates for group fairness metrics that account for network structure. More specifically, we present two types of group fairness metrics that measure distinct yet complementary forms of bias in networks. The first type of metric evaluates how access to others in the network is distributed across groups. The second type of metric evaluates how groups distribute their interactions across other groups, and hence captures inter-group biases. We find that ignoring the network can lead to spurious fairness evaluations by either not capturing imbalances in influence and reach illuminated by the first type of metric, or by overlooking interaction biases as evaluated by the second type of metric. Our empirical study illustrates these pronounced differences between network and non-network evaluations of fairness.2022AMAnay Mehrotra et al.Data, Bias and FairnessCSCW
DistanciAR: Authoring Site-Specific Augmented Reality Experiences for Remote EnvironmentsMost augmented reality (AR) authoring tools only support the author's current environment, but designers often need to create site-specific experiences for a different environment. We propose DistanciAR, a novel tablet-based workflow for remote AR authoring. Our baseline solution involves three steps. A remote environment is captured by a camera with LiDAR; then, the author creates an AR experience from a different location using AR interactions; finally, a remote viewer consumes the AR content on site. A formative study revealed understanding and navigating the remote space as key challenges with this solution. We improved the authoring interface by adding two novel modes: Dollhouse, which renders a bird's-eye view, and Peek, which creates photorealistic composite images using captured images. A second study compared this improved system with the baseline, and participants reported that the new modes made it easier to understand and navigate the remote scene.2021ZWZeyu Wang et al.Yale UniversityAR Navigation & Context AwarenessMixed Reality WorkspacesDigital Art Installations & Interactive PerformanceCHI