A Framework for Adapting In-Car Touchscreen Interfaces to Driver Behaviors, Perception, and CognitionAlthough in-car touchscreens expand interaction possibilities, they risk compromising driver safety and vigilance. We propose a data- and expert-informed framework for designing adaptive touchscreens that respond to a driver’s usage profile and cognitive state, maximizing usability while mitigating safety risks. First, in a driving simulator study, we find that cognitive load slows touchscreen button selections by 20\% and produced shorter, more frequent off-road glances. We also find that enlarging buttons improves selection speeds by 0.3 seconds but at the cost of requiring more display pages. Next, these findings informed a co-design session with expert in-cabin designers, generating guidelines for adaptive interfaces that balance usability and safety. These guidelines form the basis of our Profile-State Adaptive (PSA) framework, which integrates driver profiles with cognitive states to guide interface adaptations. We then extend the framework to include a quantitative Time-Cost model as well as design patterns for adaptive layouts across usage profiles and cognitive demands.2026SHSeokhyun Hwang et al.University of WashingtonAutomated Driving Interface & Takeover DesignHead-Up Display (HUD) & Advanced Driver Assistance Systems (ADAS)In-Vehicle Haptic, Audio & Multimodal FeedbackCHI
Touchscreens in Motion: Quantifying the Impact of Cognitive Load on Distracted DriversThis study investigates the interplay between a driver's cognitive load, touchscreen interactions, and driving performance. Using an N-back task to induce four levels of cognitive load, we measured physiological responses (pupil diameter, electrodermal activity), subjective workload (NASA-TLX), touchscreen performance (Fitts' law), and driving metrics (lateral deviation, throttle control). Our results reveal significant mutual performance degradation, with touchscreen pointing throughput decreasing by over 58.1% during driving conditions and lateral driving deviation increasing by 41.9% when touchscreen interactions were introduced. Under high cognitive load, participants demonstrated a 20.2% increase in pointing movement time, 16.6% decreased pointing throughput, and 26.3% reduced off-road glance durations. We identified a prevalent "hand-before-eye" phenomenon where ballistic hand movements frequently preceded visual attention shifts. These findings quantify the impact of cognitive load on multitasking performance and demonstrate how drivers adapt their visual attention and motor-visual coordination when cognitive resources are constrained.2025XSXiyuan Shen et al.Head-Up Display (HUD) & Advanced Driver Assistance Systems (ADAS)In-Vehicle Haptic, Audio & Multimodal FeedbackUIST
Buoyancé: Reeling Helium-Inflated Balloons with Mobile Robots on the Ground for Mid-Air Tangible Display, Interaction, and AssemblyWe introduce a novel approach to spatially actuated tangible UI by controlling helium-inflated balloons (HIBs) in mid-air using mobile reeling robots, named ReelBots. With a relatively compact device form factor, the robots can manipulate HIBs in an extensive vertical range, reaching relatively high altitudes (20m or more), thanks to its reeling mechanisms. The hardware offers diverse interactive functionalities and applications for representing abstract data in 3D space, reconfiguring lights and cameras in an everyday space, and assembling HIBs into diverse configurations. Our proof-of-concept implementation was developed based on omnidirectional mobile robots and a motion tracking system to demonstrate the novel approach of enriching 3D physical space. Our control software is designed to manipulate multiple robots to control the position of HIBs in real time via multiple options ranging from GUI control and tangible and gesture based controls.2025APAlan Pham et al.Shape-Changing Interfaces & Soft Robotic MaterialsDigital Art Installations & Interactive PerformanceUIST
SOH Illusion: Misunderstandings of EV Battery State of Health and Methods to Promote UnderstandingLegislation in the USA will soon require that electric vehicles (EVs) display battery degradation in the instrument cluster as "state of health" (SOH), the percentage of the battery's original capacity. However, the extent to which consumers understand SOH degradation patterns is not known. In an initial study with vehicle owners, we find preliminary evidence for a 'SOH illusion', wherein people expect linear rates of EV battery degradation over time even though batteries degrade non-linearly. Additionally, a third of participants incorrectly conflated SOH with a battery's remaining usable life, demonstrating some misunderstanding of SOH among vehicle owners. In a follow-up study we find that framing SOH information with different chart types and legends reduces linear degradation assumptions and aligns people's expectations. We discuss implications for the design of SOH representations in user interfaces that vehicle UI designers could employ to promote better EV battery understanding.2025KLKylie R. Lin et al.EV Charging & Eco-Driving InterfacesInteractive Data VisualizationAutoUI
From Dashboards to Dialogue: Evaluating a Conversational AI Coach for Performance Driving Skill DevelopmentLearning in domains involving complex motor skills, such as performance driving, often requires feedback that is timely, personalized, and actionable. Yet many drivers rely on video and telemetry data to review their performance without guidance. We explore how conversational AI can support post-drive reflection by integrating LLM-generated coaching into an interactive review interface. In an exploratory within-subjects simulator study (n=16), participants completed laps under two conditions: one with video and data visualizations alone, and another with the same tools augmented with a conversational interface that provided verbal feedback after each lap. Conversational feedback supported short-term improvements in lap time, average speed, and steering control, and was rated as more useful and satisfying—though it also elicited slightly higher nervousness. These results suggest that conversational AI can make post-drive feedback more interpretable and actionable, particularly for drivers reviewing performance data in high-skill contexts like performance driving.2025JCJean Costa et al.Human-LLM CollaborationAI-Assisted Decision-Making & AutomationAutoUI
Research as Care: A reflection on incorporating the ethics of care in design research with people living with dementiaWhen computing researchers design technologies for vulnerable populations and engage with them over extended periods, researchers may incorporate "care"—deliberate actions to build and maintain relationships with participants—to improve engagement and deepen their understanding of situated perspectives. However, when researchers choose to take actions involving care, these efforts are rarely made explicit. Reflecting on our three-year project of designing and testing a social robot with 31 participants living with dementia, we realized the benefit of intentional reflection on the ethics and practice of care during the research process. We offer "research as care" guidelines into computing design research: 1) viewing participants as individuals, 2) being intentional in the ongoing and dynamic engagement, 3) acknowledging the reciprocity inherent in care, 4) reporting care practices transparently, 5) tailoring care to the specific context, and 6) making an informed choice to incorporate care. By incorporating research as care, computing design researchers can provide a more productive experience for participants and enhance their designs' overall quality and validity.2025LHLong-Jing Hsu et al.Elderly Care & Dementia SupportSocial Robot InteractionParticipatory DesignDIS
BioSpark: Beyond Analogical Inspiration to LLM-augmented TransferWe present BioSpark, a system for analogical innovation designed to act as a creativity partner in reducing the cognitive effort in finding, mapping, and creatively adapting diverse inspirations. While prior approaches have focused on initial stages of finding inspirations, BioSpark uses LLMs embedded in a familiar, visual, Pinterest-like interface to go beyond inspiration to supporting users in identifying the key solution mechanisms, transferring them to the problem domain, considering tradeoffs, and elaborating on details and characteristics. To accomplish this BioSpark introduces several novel contributions, including a tree-of-life enabled approach for generating relevant and diverse inspirations, as well as AI-powered cards including 'Sparks' for analogical transfer; 'Trade-offs' for considering pros and cons; and 'Q&A' for deeper elaboration. We evaluated BioSpark through workshops with professional designers and a controlled user study, finding that using BioSpark led to a greater number of generated ideas; those ideas being rated higher in creative quality; and more diversity in terms of biological inspirations used than a control condition. Our results suggest new avenues for creativity support tools embedding AI in familiar interaction paradigms for designer workflows.2025HKHyeonsu B Kang et al.Carnegie Mellon University, Human-Computer Interaction InstituteHuman-LLM CollaborationCreative Collaboration & Feedback SystemsCHI
Bittersweet Snapshots of Life: Designing to Address Complex Emotions in a Reminiscence Interaction between Older Adults and a RobotHuman-Computer Interaction and Human-Robot Interaction researchers have developed various reminiscence technologies for older adults, but the focus of such work has mostly been on making the technology usable and improving older adults' memory recall. Our study of a robot facilitating reminiscence through conversations about personal photographs with 20 older adults uncovered a less discussed aspect of such interactions: reminiscence can evoke both \textit{bitter} and \textit{sweet} emotions. Without adequate emotional sensitivity, the robot sometimes responded inappropriately, requiring researchers to intervene in the interaction to address misunderstandings. To understand how to better address these challenges, we conducted a follow-up co-design workshop with 7 older adults to explore how the robot could better support managing bittersweet emotions. Through reflexive thematic analysis of the two studies, this paper identifies factors that trigger bittersweet emotions during reminiscence with a robot and provides strategies for technology to manage these emotions during such interactions. This research highlights the importance of addressing emotional experiences in the design of reminiscence technology. It also raises ethical concerns about the emotional vulnerability of deploying one-on-one AI technologies for older adults.2025LHLong-Jing Hsu et al.Indiana University Bloomington, InformaticsEV Charging & Eco-Driving InterfacesCognitive Impairment & Neurodiversity (Autism, ADHD, Dyslexia)Social Robot InteractionCHI
Inkspire: Supporting Design Exploration with Generative AI through Analogical SketchingWith recent advancements in the capabilities of Text-to-Image (T2I) AI models, product designers have begun experimenting with them in their work. However, T2I models struggle to interpret abstract language and the current user experience of T2I tools can induce design fixation rather than a more iterative, exploratory process. To address these challenges, we developed Inkspire, a sketch-driven tool that supports designers in prototyping product design concepts with analogical inspirations and a complete sketch-to-design-to-sketch feedback loop. To inform the design of Inkspire, we conducted an exchange session with designers and distilled design goals for improving T2I interactions. In a within-subjects study comparing Inkspire to ControlNet, we found that Inkspire supported designers with more inspiration and exploration of design ideas, and improved aspects of the co-creative process by allowing designers to effectively grasp the current state of the AI to guide it towards novel design intentions.2025DLDavid Chuan-En Lin et al.Carnegie Mellon University, Human-Computer Interaction InstituteGenerative AI (Text, Image, Music, Video)Creative Collaboration & Feedback SystemsCustomizable & Personalized ObjectsCHI
VIME: Visual Interactive Model Explorer for Identifying Capabilities and Limitations of Machine Learning Models for Sequential Decision-MakingEnsuring that Machine Learning (ML) models make correct and meaningful inferences is necessary for the broader adoption of such models into high-stakes decision-making scenarios. Thus, ML model engineers increasingly use eXplainable AI (XAI) tools to investigate the capabilities and limitations of their ML models before deployment. However, explaining sequential ML models, which make a series of decisions at each timestep, remains challenging. We present Visual Interactive Model Explorer (VIME), an XAI toolbox that enables ML model engineers to explain decisions of sequential models in different ``what-if'' scenarios. Our evaluation with 14 ML experts, who investigated two existing sequential ML models using VIME and a baseline XAI toolbox to explore ``what-if'' scenarios, showed that VIME made it easier to identify and explain instances when the models made wrong decisions compared to the baseline. Our work informs the design of future interactive XAI mechanisms for evaluating sequential ML-based decision support systems.2024AAAnindya Das Antar et al.Eye Tracking & Gaze InteractionExplainable AI (XAI)AI-Assisted Decision-Making & AutomationUIST
Gliding on Simulated Ice: Effect of Low-μ Emulation on Drift TrainingDrifting, a skillful driving technique involving intentional traction loss and counter-steering, traditionally demands high-speed maneuvers under high-friction conditions, posing significant risks and fear for novices. Our study explores low-{\textmu} (low friction) emulation, simulating icy conditions to facilitate drift training at safer, lower speeds. This approach not only enhances safety and mitigates fear by reducing the required speed for drifting, but also extends the time for them to react. A between-group design was employed, comparing drift training outcomes between participants trained exclusively in higher-{\textmu} conditions (control group) and those who trained initially in lower-{\textmu} conditions before transitioning to higher-{\textmu} conditions (target group). The performance was assessed through the average distance of continuous sliding, along with subjective measures of motivation and workload. The results showed that the target group achieved greater slide distances in the retention session and reported higher scores on the positive intrinsic motivation factors, suggesting enhanced performance and engagement.2024HYHiroshi Yasuda et al.Automated Driving Interface & Takeover DesignMotion Sickness & Passenger ExperienceAutoUI
On Stress: Combining Human Factors and Biosignals to Inform the Placement and Design of a Skin-like Stress SensorWith advances in electronic-skin and wearable technologies, it is possible to continuously measure stress markers from the skin and sweat to monitor and improve wellbeing and health. Understandably, the sensor's engineering and resolution are important towards its function. However, we find that people looking for an e-skin stress sensor may look beyond measurement precision, demanding a private and stealth design to reduce, for example, social stigmatization. We introduce the idea of a stress sensing "wear index," created from the combination of human-centered design (n=24), physiological (n=10), and biochemical (n=16) data. This wear index can inform the design of stress wearables to fit specific applications, e.g., human factors may be relevant for a wellbeing application, versus a relapse prevention application that may require more sensing precision. Our wear index idea can be further generalized as a method to close gaps between design and engineering practices.2024YKYasser Khan et al.University of Southern CaliforniaHaptic WearablesSleep & Stress MonitoringCHI
"Give it Time": Longitudinal Panels Scaffold Older Adults' Learning and Robot Co-DesignParticipatory robot design projects with older adults often use multiple sessions to encourage design feedback and active participation from users. These projects have, however, not analyzed the learning outcomes for older adults across co-design sessions and how they support constructive design feedback and meaningful participation. To bridge this gap, we examined the learning outcomes within a "longitudinal panel." This panel comprised seven co-design sessions with 11 older adults of varying cognitive abilities over six months, aimed at designing a robot to guide a photograph-based conversational activity. Using Nelson and Stolterman's framework of the hierarchy of design-learning, we demonstrate how older adult panelists achieved multiple design-learning outcomes -- capacity, confidence, capability, competence, courage, and connection -- which allowed them to provide actionable design suggestions. We provide guidelines for conducting longitudinal panels that can enhance user design-learning and participation in robot design.2024LHLong-Jing Hsu et al.Domestic RobotsParticipatory DesignHRI
Promoting Sustainable Charging Through User Interface InterventionsWith the rising popularity of electrified vehicles, emphasis has been placed on encouraging charging with renewable energy and maximizing battery longevity to improve vehicle sustainability. Many mobile applications offer tools to suggest charging times with more sustainable renewable energy and charging strategies that preserve battery health. However, these options often result in longer, less convenient charging times for drivers. Here we conducted three charging scenario studies to identify factors that influence willingness to wait for sustainable charging. Participants selected between faster but less sustainable charging options and slower charging options that either reduce charging emissions or improve battery longevity. We find people's willingness to wait for green energy is influenced by situational factors; further we find that information and battery longevity interventions can increase willingness to wait for sustainable charging. Finally, we provide design recommendations to promote sustainably in charging behaviors.2023AFAlexandre L. S. Filipowicz et al.EV Charging & Eco-Driving InterfacesSustainable HCIAutoUI
Co-designing Social Robots with People Living with Dementia: Fostering Identity, Connectedness, Security, and AutonomyConventional co-design methods, such as storyboarding and focus groups, are not always appropriate for people living with dementia (PLwD). In pilot robot co-design workshops in a local memory care facility, we noticed PLwD struggled to understand, express themselves, fully participate, and benefit from the experience. After reflecting on challenges with the facility's director of program development and education, we redesigned the workshops prioritizing elements of the Eden Alternative's well-being for PLwD: identity, connectedness, security, and autonomy. We delivered these new workshops over five weeks with 12 PLwD participants. Analysis of resulting video recordings and transcripts shows the new activities allowed participants to see themselves as having knowledge relevant to social robot design; to relate to each other, the robot, and the researchers; to feel comfortable; and to actively contribute to and offer valuable insights for robot design. Participants reported feeling meaning, growth, and joy during the workshops.2023LHLong-Jing Hsu et al.Social Robot InteractionEmpowerment of Marginalized GroupsParticipatory DesignDIS
Expanded Situational Awareness Without Vision: A Novel Haptic Interface for Use in Fully Autonomous VehiclesThis work presents a novel ultrasonic haptic interface to improve nonvisual perception and situational awareness in applications such as fully autonomous vehicles. User study results (n=14) suggest comparable performance with the dynamic ultrasonic stimuli versus a control using static embossed stimuli. The utility of the ultrasonic interface is demonstrated with a prototype autonomous small-scale robot vehicle using intersection abstractions. These efforts support the application of ultrasonic haptics for improving nonvisual information access in autonomous transportation with strong implications for people who are blind and visually impaired, accessibility, and human-in-the-loop decision making.2023PFPaul D. S. Fink et al.Automated Driving Interface & Takeover DesignMid-Air Haptics (Ultrasonic)HRI
Save A Tree or 6 kg of CO2? Understanding Effective Carbon Footprint Interventions for Eco-Friendly Vehicular ChoicesFrom ride-hailing to car rentals, consumers are often presented with eco-friendly options. Beyond highlighting a "green" vehicle and CO2 emissions, CO2 equivalencies have been designed to provide understandable amounts; we ask which equivalencies will lead to eco-friendly decisions. We conducted five ride-hailing scenario surveys where participants picked between regular and eco-friendly options, testing equivalencies, social features, and valence-based interventions. Further, we tested a car-rental embodiment to gauge how an individual (needing a car for several days) might behave versus the immediate ride-hailing context. We find that participants are more likely to choose green rides when presented with additional information about emissions; CO2 by weight was found to be the most effective. Further, we found that information framing - be it individual or collective footprint, positive or negative valence - had an impact on participants’ choices. Finally, we discuss how our findings inform the design of effective interventions for reducing car-based carbon-emissions.2023VMVikram Mohanty et al.Virginia TechEV Charging & Eco-Driving InterfacesSustainable HCIEnergy Conservation Behavior & InterfacesCHI
Understanding People's Perception and Usage of Plug-in Electric HybridsElectrification is an important first step toward reducing the greenhouse emissions of passenger vehicles. However, how drivers drive, charge, and operate their electrified vehicles can have a large impact on their emissions, particularly for Plug-in Hybrid Electric vehicles (PHEVs) that combine all-electric driving with an internal combustion engine. In this paper, we investigate how and why drivers use their PHEVs and uncover design opportunities for interfaces that can support the efficient use of PHEVs. We used a mixed-method approach combining quantitative, qualitative, and concept elicitation methods with PHEV owners in the US. While past findings indicate that PHEV drivers are not motivated to charge regularly, our work contradicts this with evidence of (1) regular charging with home infrastructure, (2) high cost sensitivity, and (3) preference for driving in all-electric mode. Our results indicate that the most critical problem is inadequate user support for navigating poor charging infrastructure.2023MLMatthew L Lee et al.Toyota Research InstituteEV Charging & Eco-Driving InterfacesCHI
Autonomous is Not Enough: Designing Multisensory Mid-Air Gestures for Vehicle Interactions Among People with Visual ImpairmentsShould fully autonomous vehicles (FAVs) be designed inclusively and accessibly, independence will be transformed for millions of people experiencing transportation-limiting disabilities worldwide. Although FAVs hold promise to improve efficient transportation without intervention, a truly accessible experience must enable user input, for all people, in many driving scenarios (e.g., to alter a route or pull over during an emergency). Therefore, this paper explores desires for control in FAVs among (n=23) people who are blind and visually impaired. Results indicate strong support for control across a battery of driving tasks, as well as the need for multimodal information. These findings inspired the design and evaluation of a novel multisensory interface leveraging mid-air gestures, audio, and haptics. All participants successfully navigated driving scenarios using our gestural-audio interface, reporting high ease-of-use. Contributions include the first inclusively designed gesture set for FAV control and insight regarding supplemental haptic and audio cues.2023PFPaul D. S. Fink et al.The University of Maine, University of MaineAutomated Driving Interface & Takeover DesignIn-Vehicle Haptic, Audio & Multimodal FeedbackVisual Impairment Technologies (Screen Readers, Tactile Graphics, Braille)CHI
You Complete Me: Human-AI Teams and Complementary ExpertisePeople consider recommendations from AI systems in diverse domains ranging from recognizing tumors in medical images to deciding which shoes look cute with an outfit. Implicit in the decision process is the perceived expertise of the AI system. In this paper, we investigate how people trust and rely on an AI assistant that performs with different levels of expertise relative to the person, ranging from completely overlapping expertise to perfectly complementary expertise. Through a series of controlled online lab studies where participants identified objects with the help of an AI assistant, we demonstrate that participants were able to perceive when the assistant was an expert or non-expert within the same task and calibrate their reliance on the AI to improve team performance. We also demonstrate that communicating expertise through the linguistic properties of the explanation text was effective, where embracing language increased reliance and distancing language reduced reliance on AI.2022QZQiaoning Zhang et al.University of Michigan-Ann ArborHuman-LLM CollaborationAI-Assisted Decision-Making & AutomationCHI