Understanding Roboticists' Power through Matrix Guided Power AnalysisRoboticists wield substantial power through the ways we choose to design and deploy robots. But understanding the nature of this power requires us to consider the different types of power wielded through different types of robot design choices, and the social and historical factors that shape the power landscape into which robots are embedded. To facilitate this type of analysis, I present Matrix-Guided Power Analysis (MGPA), a framework for analyzing the different types of power that technologists wield across different domains of power, with sensitivity to the social and historical forces that determine the default and alternative trajectories of those technologies. Further, I show how MGPA can be used to better understand the specific types of power that roboticists wield.2024TWTom WilliamsMental Health Apps & Online Support CommunitiesHuman-Robot Collaboration (HRC)Technology Ethics & Critical HCICHI
The Purr-suit of Happiness: A Tale of Three Kittens. Robots, Humans, Cats, and AIThis paper showcases Cat Royale, an exploration of the impact of artificial intelligence (AI) on animal happiness situated at the intersection of Art, Computer Science, and Animal Welfare. We argue for the inclusion of non-human actors when designing autonomous systems, as animals increasingly interact with them. In this endeavour, we emphasise multidisciplinarity when designing trustworthy autonomous system. To design, implement, and deploy such systems, diverse voices must be heard. Finally, by highlighting parallels between Cat Royale’s animal-robot interactions and human-AI interactions, this project invites reflections on the trustworthiness, risks, and the price we might pay for AI.2024ESEike Schneiders et al.Social Robot InteractionHuman-Nature Relationships (More-than-Human Design)CHI
Intelligent humanoids in manufacturing∗ Addressing the worker shortage and skill gaps in assembly cellsTechnological evolution in the field of robotics is emerging with major breakthroughs in recent years. This was especially fostered by revolutionary new software applications leading to humanoid robots. Humanoids are being envisioned for manufacturing applications to form human-robot teams. But their implication in the manufacturing practices especially for industrial safety standards and lean manufacturing practices have been minimally addressed. Humanoids will also be competing with conventional robotic arms and effective methods to assess their return on investment are needed. To study the next generation of industrial automation, we used the case context of Tesla’s humanoid robot. The company has recently unveiled its project on an intelligent humanoid robot named ‘Optimus’ to achieve an increased level of manufacturing automation. This article proposes a framework to integrate humanoids for manufacturing automation and also presents the significance of safety standards of human-robot collaboration. A case of lean assembly cell for the manufacturing of an open source medical ventilator was used for human-humanoid automation. Simulation results indicate that humanoids can increase the level of manufacturing automation. Managerial and research implications are presented.2024AMAli Ahmad MalikHuman-Robot Collaboration (HRC)Warehouse & Industrial RobotsImpact of Automation on WorkCHI
A Case for Diverse Social Robot Identity Performance in EducationEducational outcomes for students belonging to disadvantaged social identities are unavoidably influenced by overlapping systems of inequity which arise along lines such as gender, ethnicity, and age. Robot platforms such as Furhat require designers to select features which are interpreted by users as these same kinds of social identity. Prior work has posited that social robots might be intentionally designed to leverage these social identities in a "norm-breaking" fashion with the aim of disrupting social stereotypes in STEM education. However, research in HRI has been largely limited to the examination of gender only. We present a 2x2, between-subjects study in which 161 participants aged 9-12 are shown a robot-delivered lecture presented by a group of three separate robot personas with varying gender and ethnicity performances. We find that participants place greater trust in the persona groups with high gender diversity. Incorporating ethnic diversity seems to have little impact on our quantitative interaction metrics, however we do find evidence to suggest diversity in robots' language capabilities may be important for trustworthiness. In all, the study contributes nuance to the discussions on the implications of (norm-breaking) social identity performance when using robots to pursue more equitable STEM education.2024LMLux Miranda et al.Social Robot InteractionRobots in Education & HealthcareCommunity Engagement & Civic TechnologyCHI
Theory of Mind abilities of Large Language Models in Human-Robot Interaction : An Illusion?Large Language Models have shown exceptional generative abilities in various natural language and generation tasks. However, possible anthropomorphization and leniency towards failure cases have propelled discussions on emergent abilities of Large Language Models especially on Theory of Mind (ToM) abilities in Large Language Models. While several false-belief tests exists to verify the ability to infer and maintain mental models of another entity, we study a special application of ToM abilities that has higher stakes and possibly irreversible consequences : Human Robot Interaction. In this work, we explore the task of Perceived Behavior Recognition, where a robot employs a Large Language Model (LLM) to assess the robot's generated behavior in a manner similar to human observer. We focus on four behavior types, namely - explicable, legible, predictable, and obfuscatory behavior which have been extensively used to synthesize interpretable robot behaviors. The LLMs goal is, therefore to be a human proxy to the agent, and to answer how a certain agent behavior would be perceived by the human in the loop, for example "Given a robot's behavior X, would the human observer find it explicable?". We conduct a human subject study to verify that the users are able to correctly answer such a question in the curated situations (robot setting and plan) across five domains. A first analysis of the belief test yields extremely positive results inflating ones expectations of LLMs possessing ToM abilities. We then propose and perform a suite of perturbation tests which breaks this illusion, i.e. Inconsistent Belief, Uninformative Context and Conviction Test. We conclude that, the high score of LLMs on vanilla prompts showcases its potential use in HRI settings, however to possess ToM demands invariance to trivial or irrelevant perturbations in the context which LLMs lack. We report our results on GPT-4 and GPT-3.5-turbo.2024MVMudit Verma et al.Electrical Muscle Stimulation (EMS)Brain-Computer Interface (BCI) & NeurofeedbackHuman-LLM CollaborationCHI
More-than-human Perspective on the Robomorphism ParadigmThis paper proposes a posthuman perspective of the robomorphism theory. We propose to define robomorphism as the attribution of robotlike traits to non-robotic entities. Such a definition embraces the centrality of robots in two aspects. First, by assuming the target of robomorphism is not necessarily a human. Second, by considering the notion of robomorphic traits as inherently crucial to establish the robomorphism paradigm. Embracing robots as relevant non-humans in the robomorphism paradigm constitutes the more-than-human perspective of the proposed approach. The contributions of this paper are threefold. First, we propose the robomorphism paradigm by defining it and its inherent concepts, such as robomorphisation and robomorphic. Second, we discuss the broader implications of the robomorphism theory to the research community of Human-Robot Interaction, raising important new challenges. Third, we created a preliminary inventory of robomorphic traits, which were collected from a speculative workshop activity in order to start answering one of the proposed open challenges.2024FCFilipa Correia et al.Human-Robot Collaboration (HRC)Technology Ethics & Critical HCIHuman-Nature Relationships (More-than-Human Design)CHI
The Human Behind the Robot: Rethinking the Low Social Status of Service RobotsRobots in our society are commonly perceived as subordinate servants with a lower social status than humans. This often leads to humans prioritizing themselves during conflict situations. This becomes problematic when robots start to directly represent humans as proxies if people do not think of the human operator behind them. This could be considered a cognitive bias of human representation in HRI. To explore the extent of this problem, we conducted a user study featuring several conflict situations. Participants granted more priority to the robot when the human representation was visible. This paper explores the societal consequences and emerging inequities such as potentially deprioritizing humans by deprioritizing a robot in certain situations. Possible strategies to address potential negative consequences are discussed on a design level while acknowledging that a societal change in how we perceive and treat robots that represent humans might be necessary.2024FBFranziska Babel et al.Privacy by Design & User ControlSocial Robot InteractionHuman-Robot Collaboration (HRC)CHI
Spatial Robotic Experiences as a Ground for Future HRI SpeculationsThis work illustrates how artistic robotic systems can provide a reservoir of unfamiliarity and a basis for speculation, to open the field toward new ways of thinking about HRI. We reflect on a collaborative project between design students, a media art studio, and design researchers working with the baggage handling department of a strategic European airport. Engaging with the industrial context, we developed ‘meta-behaviours’ - abstracted ideas of processes carried out on the worksite–and passed these over to the students who translated them into robotic enactions based on hardware and a form language developed by the media art studio. The resulting visit experience challenges the audience to decode the installation in terms of meta-behaviours and their possible relations to industrial HRI. We used this to reflect on the value of conducting artistic and speculative work in HRI and to distil actionable recommendations for future research.2024DMDave Murray-Rust et al.Human-Robot Collaboration (HRC)Technology Ethics & Critical HCIDesign FictionCHI
User-Designed Human-UAV Interaction in a Social Indoor EnvironmentThe purpose of this project is to understand how people would expect to interact with an Unmanned Aerial Vehicle (UAV) in a social indoor environment under friendly, neutral, or adversarial contexts. The three environments will include one setting with the UAV serving as a tour guide, one as a security guard, and one as a food delivery mechanism. This work is novel in its inquiry into the affective nature of the interaction, comparison across situational contexts, and ability to compare preferences both within and between participants. Our findings will help researchers plan for appropriate safety and comfort measures, while being cognizant of the participants’ preferences for and understanding of how drones operate. This study examines realistic indoor scenarios for which each participant designs their preferred interaction and presents exploratory results, including comparison to prior work with respect to motion gestures and comfortable approach distances. Initial findings suggest the importance of visibility of approaches, selecting approach heights relative to the person and based on the context of interaction, and criticality of the initial direction of motion when classifying the communicative content of UAV flight paths.2024ABAlisha Bevins et al.Drone Interaction & ControlCHI
A Virtual Reality Framework for Human-Driver Interaction Research: Safe and Cost-Effective Data CollectionThe advancement of automated driving technology has led to new challenges in the interaction between automated vehicles and human road users. However, there is currently no complete theory that explains how human road users interact with vehicles, and studying them in real-world settings is often unsafe and time-consuming. This study proposes a 3D Virtual Reality (VR) framework for studying how pedestrians interact with human-driven vehicles and autonomous vehicles. The framework uses VR technology to collect data in a safe and cost-effective way, and deep learning methods are used to predict pedestrian trajectories. Specifically, graph neural networks have been used to model pedestrian future trajectories and the probability of crossing the road. The results of this study show that the proposed framework can be for collecting high-quality data on pedestrian-vehicle interactions in a safe and efficient manner. The data can then be used to develop new theories of human-vehicle interaction and to train autonomous vehicles to better interact with pedestrians.2024LCLuca Crosato et al.External HMI (eHMI) — Communication with Pedestrians & CyclistsV2X (Vehicle-to-Everything) Communication DesignCHI
What is your other hand doing, robot? A model of behavior for shopkeeper robot's idle handIn retail settings, a robot's one-handed manipulation of objects can come across as thoughtless and impolite, thus creating a negative customer experience. To solve this problem, we first observed how human shopkeepers interact with customers, specifically focusing on their hand movements during object manipulation. From the observation and analysis of shopkeepers' hand movements, we identified an essential element of their idle hand movements: "support" provided by the idle hand as the primary hand manipulates an object. Based on this observation, we proposed a model that coordinates the movements of a robot's idle hand with its primary task-engaged hand, emphasizing its supportive behaviors. In a within-subjects study, 20 participants interacted with robot shopkeepers under different conditions to assess the impact of incorporating support behavior with the idle hand. The results show that the proposed model significantly outperforms a baseline in terms of politeness and competence, suggesting enhanced object-based interactions between the robot shopkeepers and customers.2024XPXiang Pan et al.Human-Robot Collaboration (HRC)CHI
Enhancing Safety in Learning from Demonstration Algorithms via Control Barrier Function ShieldingLearning from Demonstration (LfD) is a powerful method for non-roboticists end-users to teach robots new tasks, enabling them to customize the robot behavior. However, modern LfD techniques do not explicitly synthesize safe robot behavior, which limits the deployability of these approaches in the real world. To enforce safety in LfD without relying on experts, we propose a new framework, ShiElding with Control barrier fUnctions in inverse REinforcement learning (SECURE), which learns a customized Control Barrier Function (CBF) from end-users that prevents robots from taking unsafe actions while imposing little interference with the task completion. We evaluate SECURE in three sets of experiments. First, we empirically validate SECURE learns a high-quality CBF from demonstrations and outperforms conventional LfD methods on simulated robotic and autonomous driving tasks with improvements on safety by up to 100%. Second, we demonstrate that roboticists can leverage SECURE to outperform conventional LfD approaches on a real-world knife-cutting, meal-preparation task by 12.5% in task completion while driving the number of safety violations to zero. Finally, we demonstrate in a user study that non-roboticists can use SECURE to effectively teach the robot safe policies that avoid collisions with the person and prevent coffee from spilling.2024YYYue Yang et al.AI-Assisted Decision-Making & AutomationHuman-Robot Collaboration (HRC)CHI
Modelling Experts' Sampling Strategy to Balance Multiple Objectives During Scientific ExplorationsDuring scientific explorations, scientists often hold multiple and often conflicting objectives. Understanding how scientists prioritize and balance these objectives is crucial for developing cognitively-compatible robotic teammates and fostering effective human-robot collaboration. In this study, we seek to improve the cognitive compatibility of robotic algorithms by modelling human' decision making processes under multiple objectives. Collected human decision data from 141 sampling steps indicate that the majority of scientists adopt one of the following objective balancing strategies: (i) A Focus mode, where experts select sampling location to primarily optimize their primary objective; (ii) A Hierarchy mode, where experts hierarchically satisfy foremost their primary objective, then, to a lesser extent, their secondary objective; and (iii) A Trade-off mode, where experts select sampling locations to satisfy all objectives, even the location was not ideal for either objective. To understand how experts choose among the different modes, we quantitatively characterize the three types of strategies, by representing the decision data from each sampling step in an objective function space. Analysis of the strategy types reveals that, experts' adaptation of multi-objective coordinating strategies are primarily governed by two key decision factors: current stages of sampling, and outstanding reward values. This discovery allows the robot to use an extremely simple decision algorithm to connect experts' high-level objectives to desired sampling locations when balancing multiple objectives. Deployment of this algorithm at a planetary-analogue field exploration mission on Mt. Hood demonstrates the potential for robots to use cognitively-compatible algorithms to participate in decision making and aid with the adaptation of sampling plans that align with scientists' high-level goals.2024SLShipeng Liu et al.Human-LLM CollaborationAI-Assisted Decision-Making & AutomationComputational Methods in HCICHI
Human perception of swarm fragmentationIn the context of robot swarms, fragmentation refers to a breakdown in communication and coordination among the robots. This fragmentation can lead to issues in the swarm self-organisation, especially the loss of efficiency or an inability to perform their tasks. Human operators influencing the swarm could prevent fragmentation. To help them in this task, it is necessary to study the ability of humans to perceive and anticipate fragmentation. This article studies the perception of different types of fragmentation occurring in swarms depending on their behaviour selected amongst swarming, flocking, expansion and densification. Thus, we characterise human perception thanks to two metrics based on the distance separating fragmented groups and the separation speed. The experimentation protocol consists of a binary discrimination task in which participants have to assess the presence of fragmentation. The results show that detecting fragmentation for expansion behaviour and anticipating fragmentation, in general, are challenging. Moreover, they show that humans rely on separation distance and speed to infer the presence or absence of fragmentation. Our study paves the way for new research that will provide information to humans to better anticipate and efficiently prevent the occurrence of swarm fragmentation.2024AHAymeric Hénard et al.Human Pose & Activity RecognitionHuman-Robot Collaboration (HRC)CHI
Towards human-like handover timing performance with legged manipulatorsDeploying perception modules for human-robot handovers is challenging because they require a high degree of reactivity, generalizability, and robustness to work reliably for a diversity of objects. Further complications arise as each object can be handed over in a variety of ways, that can cause occlusions and viewpoint changes. On legged robots, deployment is particularly challenging because of the limited computational resources and the additional image-space noise resulting from locomotion. In this paper, we introduce an efficient and object-agnostic real-time tracking framework, specifically designed for handover tasks between a human and a legged manipulator. The proposed method combines fast optical flow with Siamese-network-based tracking and depth segmentation in an adaptive Kalman Filter framework. We show that we outperform the state-of-the-art for tracking during human-robot handovers with our legged manipulator system. We demonstrate the generalizability, reactivity, and robustness of our system through experiments in different handover scenarios and by carrying out a user study. Furthermore, as timing has been proven to be more important than spatial accuracy in human-robot interaction tasks, we show that we reach close to human timing performance not only in terms of objective metrics, such as handover and reaction time but also by considering subjective metrics gathered from the participants in the user study.2024ATAndreea Tulbure et al.Human-Robot Collaboration (HRC)CHI
"Oh, sorry, I think I interrupted you": Designing Repair Strategies for Robotic Longitudinal Well-being CoachingRobotic well-being coaches have been shown to successfully promote people’s mental well-being. In order to provide successful coaching, a robotic coach should have the capability to repair the mistakes it makes. However, past works investigating robot mistakes are limited to game or task-based, one-off and in-lab studies. This paper presents a 4-phase design process to design repair strategies for robotic longitudinal well-being coaching with the involvement of real-world stakeholders. The design process consists of the following phases: 1) designing repair strategies with a professional well-being coach; 2) undertaking a longitudinal study with the involvement of experienced users (i.e., who had already interacted with a robotic coach before) to investigate the repair strategies defined in (1); 3) conducting a design workshop with the users of the study in (2) to gather their perspectives on the robotic coach’s repair strategies; 4) discussing the results obtained in (2) and (3) with the mental well-being professional to reflect on how to design repair strategies for robotic coaching. Our results show that users have different expectations for a robotic coach than a human coach, which influences how repair strategies should be designed. We show that different repair strategies (e.g., apologizing, explaining, or repairing empathically) are appropriate in different scenarios, and that preferences for repair strategies change during longitudinal, repeated interactions with the robotic coach.2024MAMinja Axelsson et al.Mental Health Apps & Online Support CommunitiesSocial Robot InteractionCHI
Effect of Social Robot’s Role and Behavior on Parent-Toddler InteractionSocial robots, designed to interact with people through natural communication modes like speech, body motion, gestures, and facial expressions, have been extensively studied in child-robot interaction for educational purposes. Recently, social robots have been explored in triadic parent-child-robot interactions, showing promise due to their interactivity, computational power, and physical presence, which enable multimodal natural communication and cater to toddlers' developmental stages and physical curiosity. However, these have focused only on shared reading experiences and engaged older children, rather than toddlers. We developed two games, one with two levels of robot scaffolding, and another with either structured or unstructured design. We then explored, in two studies, how a social robot's assigned role and behaviors influence the engagement of parents and toddlers with the robot and their interaction with each other. Our results show that parents affectively scaffolded their children less when the robot increased its scaffolding behaviors and that parents provided more scaffolding in a structured game with the robot, whereas in an unstructured game the dyad exhibited more cooperation in which children exhibited more independence. These findings can contribute to a better understanding of interaction design, triadic dynamics, and the role of the robot in parent-toddler-robot scenario.2024GGGoren Gordon et al.Social Robot InteractionCHI
Design and Evaluation of a Socially Assistive Robot Schoolwork Companion for College Students with ADHDStudies have shown that college students with ADHD respond positively to simple socially assistive robots (SARs) that monitor attention and provide non-verbal feedback while completing a task, but the explorations have been done only in lab settings for short sessions. This work presents an initial design and evaluation of an in-dorm socially assistive robot study companion for college students with ADHD. This work represents the introductory stages of an ongoing user-centered, participatory design process, motivated by, and conducted through a lens of life-long assistive technology, rather than short-term interventions. In a within-subjects user study, a group of university students (N=11) with self-reported symptoms of adult ADHD were randomly assigned to have a study companion robot system or a computer-based system placed in their dorm room for a total of three weeks. With the goal of developing long-term assistive technology, we focus our analysis on: 1) evaluating the usability and desire for SAR study companions among college students with ADHD, and 2) collecting extensive feedback from participants about the design and functionality of the robot. We show that participants did find the robot useful, demonstrated by an excellent average system usability scale (SUS) score of 83.864. Furthermore, after one week of using the robot regularly, 91\% of participants (10 out of 11) elected to continue using the study companion robot in the second week when they were not required to do so. We found that participants’ perceived usability of the robot was strongly correlated with how long they voluntarily studied with the robot.2024AOAmy O'Connell et al.Cognitive Impairment & Neurodiversity (Autism, ADHD, Dyslexia)Special Education TechnologyRobots in Education & HealthcareCHI
Towards Balancing Preference and Performance through Adaptive Personalized ExplainabilityAs robots and digital assistants are deployed in the real world, these agents must be able to communicate their decision-making criteria to build trust, improve human-robot teaming, and enable collaboration. While the field of explainable artificial intelligence (xAI) has made great strides in building a set of mechanisms to enable such communication, these advances often assume that one approach is ideally suited to each problem (e.g., decision trees for explaining how to triage patients in an emergency or feature-importance maps for explaining radiology reports). This fails to recognize that users may have different experiences or preferences for interaction modalities. In this work, we present the design and results of two user-studies set in a simulated autonomous vehicle (AV) domain, a setting that is increasingly important to HRI. We investigate (1) population-level preferences for xAI and (2) different personalization strategies for providing robot explanations. We find significant differences between xAI modes in both preference (p < 0.01) and task-performance (p < 0.05). We also observe that a participant's preferences do not always align with their task-performance, motivating our development of an adaptive personalization strategy that balances the two. We show that this strategy leads to significant performance gains (p < 0.05), and we conclude with a discussion our findings and implications for future work in xAI.2024ASAndrew Silva et al.Brain-Computer Interface (BCI) & NeurofeedbackExplainable AI (XAI)AI-Assisted Decision-Making & AutomationCHI
When Do People Want an Explanation from a Robot?Explanations are a critical topic in AI and robotics, and their importance in generating trust and allowing for successful human–robot interactions has been widely recognized. However, it is still an open question when and in what interaction contexts users most want an explanation from a robot. In our pre-registered study with 186 participants, we set out to identify a set of scenarios in which users show a strong need for explanations. Participants are shown 16 videos portraying seven distinct situation types, from successful human–robot interactions to robot errors and robot inabilities. Afterwards, they are asked to indicate if and how they wish the robot to communicate subsequent to the interaction in the video. The results provide a set of interactions, grounded in literature and verified empirically, in which people show the need for an explanation. Moreover, we can rank these scenarios by how strongly users think an explanation is necessary and find statistically significant differences. Comparing giving explanations with other possible response types, such as the robot apologizing or asking for help, we find that why-explanations are always among the two highest-rated responses, with the exception of when the robot simply acts normally and successfully. This stands in stark contrast to the other possible response types that are useful in a much more restricted set of situations. Lastly, we test for factors of an individual that might influence their response preferences, for example, their general attitude towards robots, but find no significant correlations. Our results can guide roboticists in designing more user-centered and transparent interactions and let explainability researchers develop more pinpointed explanations.2024LWLennart Wachowiak et al.Brain-Computer Interface (BCI) & NeurofeedbackAI-Assisted Decision-Making & AutomationSocial Robot InteractionCHI