GraspR: A Computational Model of Spatial User Preferences for Adaptive Grasp UI DesignGrasp User Interfaces (grasp UIs) enable dual-tasking in XR by allowing interaction with digital content while holding physical objects. However, designing grasp UIs presents a fundamental challenge: existing approaches either capture user preferences through labor-intensive elicitation studies that don't scale, or rely on biomechanical models that ignore subjective factors. We introduce GraspR, the first computational model that predicts user preferences for single-finger microgestures in grasp UIs. Our data-driven approach combines the scalability of computational methods with human preference modeling, trained on 1,520 preferences collected via a two-alternative forced choice paradigm across eight participants and four frequently used grasp variations. We demonstrate GraspR's effectiveness through a working prototype that dynamically adjusts interface layouts across four everyday tasks. We release both dataset and code to support future research in adaptive grasp UIs.2025ACArthur Caetano et al.Haptic WearablesShape-Changing Interfaces & Soft Robotic MaterialsFull-Body Interaction & Embodied InputUIST
SparseEMG: Computational Design of Sparse EMG Layouts for Sensing GesturesGesture recognition with electromyography (EMG) is a complex problem influenced by gesture sets, electrode count and placement, and machine learning parameters (e.g., features, classifiers). Most existing toolkits focus on streamlining model development but overlook the impact of electrode selection on classification accuracy. In this work, we present the first data-driven analysis of how electrode selection and classifier choice affect both accuracy and sparsity. Through a systematic evaluation of 28 combinations (4 selection schemes, 7 classifiers), across six datasets, we identify an approach that minimizes electrode count without compromising accuracy. The results show that Permutation Importance (selection scheme) with Random Forest (classifier) reduces the number of electrodes by 53.5%. Based on these findings, we introduce SparseEMG, a design tool that generates sparse electrode layouts based on user-selected gesture sets, electrode constraints, and ML parameters while also predicting classification performance. SparseEMG supports 50+ unique gestures and is validated in three real-world applications using different hardware setups. Results from our multi-dataset evaluation show that the layouts generated from the SparseEMG design tool are transferable across users with only minimal variation in gesture recognition performance.2025AKAnand Kumar et al.Electrical Muscle Stimulation (EMS)Hand Gesture RecognitionUIST
HydroHaptics: High-Fidelity Force-Feedback on Soft Deformable Interfaces using Hydrostatic TransmissionSoft deformable interfaces offer unique interaction potential through input flexibility and diverse forms. However, force feedback on these devices remains limited, with pneumatic approaches lacking responsiveness and precision, while microhydraulic solutions are constrained to small form factors with limited input. We present HydroHaptics, a novel platform that enables high-fidelity force feedback on deformable interfaces via hydrostatic transmission. Surpassing current state-of-the-art methods, our approach allows fine-grained force feedback on soft interfaces, achieving a 10 N force change in < 100 ms and accurate 1 N, 10 Hz oscillation rendering. We detail the system's design and implementation, highlighting its ability to maintain the inherent interaction benefits of soft interfaces. A user study (N = 18)evaluates the system's performance, showing high accuracy in rendering distinct haptic effects (82.6% accuracy) and classifying input gestures (89.1% accuracy). To showcase the platform’s versatility, we present four applications illustrating HydroHaptics' potential to enhance interaction with deformable devices and unlock novel user experiences.2025JNJames David Nash et al.Force Feedback & Pseudo-Haptic WeightShape-Changing Interfaces & Soft Robotic MaterialsUIST
It Sounds Squishy: Understanding Cross-Modal Correspondences of Deformable Shapes and Sounds Computing interfaces are becoming increasingly sophisticated, with systems that engage multiple sensory channels simultaneously. Deformable and shape-changing interfaces offer rich tactile experiences, but there is limited understanding of how they can be combined with other modes of sensory feedback. We systematically explored the audio, visual and tactile cross-modal correspondences of deformable shapes with a particular focus on auditory feedback. 50 participants were asked to associate deformable tactile stimuli, varying in stiffness and shape, with the sound qualities pitch, brightness, fade-in time and fade-out time, under visuo-tactile and tactile-only conditions. Our findings provide the first insights on how (1) shape, both its form and visibility, play a significant role in associations for pitch and brightness; (2) stiffness plays a dominant role in associations over a sound’s fade-in and fade-out times. These findings are distilled into the first design guidelines for integrating auditory feedback into physical interfaces.2025MPMaisie Palmer et al.In-Vehicle Haptic, Audio & Multimodal FeedbackMid-Air Haptics (Ultrasonic)Shape-Changing Interfaces & Soft Robotic MaterialsDIS
There Is More to Dwell Than Meets the Eye: Toward Better Gaze-Based Text Entry Systems With Multi-Threshold DwellDwell-based text entry seems to peak at 20 words per minute (WPM). Yet, little is known about the factors contributing to this limit, except that it requires extensive training. Thus, we conducted a longitudinal study, broke the overall dwell-based selection time into six different components, and identified several design challenges and opportunities. Subsequently, we designed two novel dwell keyboards that use multiple yet much shorter dwell thresholds: Dual-Threshold Dwell (DTD) and Multi-Threshold Dwell (MTD). The performance analysis showed that MTD (18.3 WPM) outperformed both DTD (15.3 WPM) and the conventional Constant-Threshold Dwell (12.9 WPM). Notably, absolute novices achieved these speeds within just 30 phrases. Moreover, MTD’s performance is also the fastest-ever reported average text entry speed for gaze-based keyboards. Finally, we discuss how our chosen parameters can be further optimized to pave the way toward more efficient dwell-based text entry.2025AMAunnoy K Mutasim et al.Simon Fraser University, School of Interactive Arts and TechnologyEye Tracking & Gaze InteractionMotor Impairment Assistive Input TechnologiesCHI
Investigating the Benefits of Physical Models for Anatomical Education in Augmented RealityHistorically, anatomical education has utilised physical models; researchers are now looking to Augmented Reality (AR) to deliver more engaging learning experiences. While there are clear educational advantages to AR, most systems lack the cognitive benefits afforded by physical models. Our work explores the potential of combining physical anatomical models and AR. We first present a design space exploring the interplay between the two. From this, we created a tangible AR system utilising a physical vertebrae model for learning spinal anatomy and axial spondyloarthritis progression. We conducted a study (n=39) to evaluate its benefits for knowledge improvement and retention, compared with a virtual AR and screen-based version. We found no difference in learning outcomes, however, the physical model improved participants' learning experience. We then conducted an expert evaluation with clinicians to explore opportunities for using tangible AR in clinical practice. Results highlight potential benefits for patient understanding, and challenges surrounding accessibility.2025MWMatthew Wragg et al.University of Bath, Department of Computer ScienceMixed Reality WorkspacesVR Medical Training & RehabilitationCHI
The Influence of Content Modality on Perceptions of Online MisinformationSocial media has become a primary information source, with platforms evolving from text-based to multi-modal environments that include images and videos. While richer media modalities enhance user engagement, they also increase the spread and perceived credibility of misinformation. Most interventions to counter misinformation on social media are text-based, which may lack the persuasive power of richer modalities. This study explores whether the effectiveness of misinformation correction varies by modality, and if certain modalities of misinformation are better countered by a specific correction modality. We conducted a survey-based experiment where participants rated the credibility of misinformation tweets before and after exposure to corrections, across all combinations of text, images and video modalities. Our findings suggest that corrections are most effective when their modality richness matches that of the original misinformation. We discuss factors affecting the perceived credibility of corrections and offer strategies to optimise misinformation correction.2025SGSuwani Gunasekara et al.University of Melbourne, School of Computing and Information SystemsContent Moderation & Platform GovernanceMisinformation & Fact-CheckingCHI
Emotionally Challenging Games Can Satisfy Older Adults' Psychological Needs: From Empirical Study to Design GuidelinesOlder adults often struggle to meet their psychological needs due to retirement and living alone. Recent studies suggest that games featuring emotional challenge (EC) can help fulfill basic psychological needs such as autonomy, competence, and relatedness by facilitating emotional exploration. However, it remains unclear whether older adults can benefit from EC games, whether they find this genre enjoyable, and how these games should be designed to better meet their needs. This work explores older adults’ experiences and perceptions of playing EC games through two studies. The first study involved playing Detroit: Become Human, revealing that older adults derived multifaceted psychological experiences from playing the game. The second study involved a custom-designed game scenario tailored to older adults, demonstrating that meaningful choices significantly influenced autonomy need satisfaction. Based on these findings, we offer five design guidelines for developing EC games that satisfy psychological needs of older adults.2025MZMin Zhou et al.Institute of Software, ChineseAging-Friendly Technology DesignSerious & Functional GamesCHI
RetroSketch: A Retrospective Method for Measuring Emotions and Presence in Virtual RealityVirtual Reality (VR) designers and researchers often need to measure emotions and presence as they evolve over time. The experience sampling method (ESM) is a common way to achieve this, however, ESM disrupts the experience and lacks granularity. We propose RetroSketch, a new method for measuring subjective emotions and presence in VR, where users watch back their VR experience and retrospectively sketch a plot of their feelings. RetroSketch leaves the VR experience undisturbed and yields highly granular data, including information about salient events and qualitative descriptions of their feelings. We compared RetroSketch and ESM in a large study (n=140) using five different VR experiences over one-hour sessions. Our results show that RetroSketch and ESM measures are highly correlated with each other, as well as physiological measures indicative of emotion. The correlations are robust across different VR experiences and user demographics. They also highlight the impact of ESM on users' experience.2025DPDominic Potts et al.University of Bath, Computer ScienceImmersion & Presence ResearchCHI
Beyond Vacuuming: How Can We Exploit Domestic Robots’ Idle Time?We are increasingly adopting domestic robots (e.g., Roomba) that provide relief from mundane household tasks. However, these robots usually only spend little time executing their specific task and remain idle for long periods. They typically possess advanced mobility and sensing capabilities, and therefore have significant potential applications beyond their designed use. Our work explores this untapped potential of domestic robots in ubiquitous computing, focusing on how they can improve and support modern lifestyles. We conducted two studies: an online survey (n=50) to understand current usage patterns of these robots within homes and an exploratory study (n=12) with HCI and HRI experts. Our thematic analysis revealed 12 key dimensions for developing interactions with domestic robots and outlined over 100 use cases, illustrating how these robots can offer proactive assistance and provide privacy. Finally, we implemented a proof-of-concept prototype to demonstrate the feasibility of reappropriating domestic robots for diverse ubiquitous computing applications.2025YSYoshiaki Shiokawa et al.University of Bath, Department of Computer ScienceContext-Aware ComputingDomestic RobotsCHI
GraspUI: Seamlessly Integrating Object-Centric Gestures within the Seven Phases of GraspingObjects are indispensable tools in our daily lives. Recent research has demonstrated their potential to act as conduits for digital interactions with microgestures, however, the primary focus was on situations where the hand firmly grasps an object. We introduce GraspUI, an exploratory design space of object-centric gestures within the seven distinct phases of the grasping process, spanning pre-, during, and post-grasp movements. We conducted ideation sessions with mixed-reality designers from industry and academia to explore gesture integration throughout the entire grasping process. The outcome was 38 storyboards envisioning practical applications. To evaluate the design space’s utility, we performed a video-based assessment with end-users. We then implemented an interactive prototype and quantified the overhead cost of performing proposed gestures through a secondary study. Participants reacted positively to gestures and could integrate them into existing usage of objects. To conclude, we highlight technical and usability guidelines for implementing and extending GraspUI systems.2024ASAdwait Sharma et al.Haptic WearablesShape-Changing Interfaces & Soft Robotic MaterialsHand Gesture RecognitionDIS
SeamSleeve: Robust Arm Movement Sensing through Powered StitchingDespite significant advances in interactive clothing over the past decade, e-textiles lack traditional fabric robustness and comfort. SeamSleeve provides a method for using garment seams as sensing channels while retaining the benefits of regular clothing design. We power conductive seams at low voltages to stitch together everyday fabric panels, resulting in a novel sensing mechanism capable of detecting arm movements. Our first empirical study (n=10) identifies optimal seam design and placement on the sleeve, by comparing traditional seam forms in the context of sensing capabilities. A second study (n=14) demonstrates that our minimal sensing approach is capable of successfully classifying 8 arm exercises with an accuracy of 84%. Our findings support the effectiveness of the approach in areas such as longitudinal physiotherapeutic rehabilitation beyond the clinic, enabling everyday motion capture.2024OROlivia G Ruston et al.Electronic Textiles (E-textiles)On-Skin Display & On-Skin InputDIS
Squishy, Yet Satisfying: Exploring Deformable Shapes' Cross-Modal Correspondences with Colours and EmotionsSurfaces with deformable and shape-changing properties seek to enhance and diversify tangible interactions with computing systems. However, we currently lack fundamental knowledge and user interface design principles that connect the inherent properties of deformable shapes with our human senses and cognitive associations. To address this knowledge gap, we systematically explored deformable shapes' cross-modal correspondences (CC) with colours and emotions. In our CC study, 52 participants were presented with deformable shape stimuli that varied in stiffness and angularity. They were asked to associate these stimuli with colours and emotions under (i) visuo-tactile and; (ii) tactile-only conditions. For the first time, our findings reveal (1) how stiffness level primarily influences the CC associations and; (2) that stiffness and angularity play a significant role in CC associations over the visibility of the shapes. The results were distilled into design guidelines for future deformable, shape-changing interfaces that engage specific human senses and responses.2024CSCameron Steer et al.University of BathMid-Air Haptics (Ultrasonic)Shape-Changing Interfaces & Soft Robotic MaterialsCHI
Sweating the Details: Emotion Recognition and the Influence of Physical Exertion in Virtual Reality ExergamingThere is great potential for adapting Virtual Reality (VR) exergames based on a user's affective state. However, physical activity and VR interfere with physiological sensors, making affect recognition challenging. We conducted a study (n=72) in which users experienced four emotion inducing VR exergaming environments (happiness, sadness, stress and calmness) at three different levels of exertion (low, medium, high). We collected physiological measures through pupillometry, electrodermal activity, heart rate, and facial tracking, as well as subjective affect ratings. Our validated virtual environments, data, and analyses are openly available. We found that the level of exertion influences the way affect can be recognised, as well as affect itself. Furthermore, our results highlight the importance of data cleaning to account for environmental and interpersonal factors interfering with physiological measures. The results shed light on the relationships between physiological measures and affective states and inform design choices about sensors and data cleaning approaches for affective VR.2024DPDominic Potts et al.University of BathImmersion & Presence ResearchFitness Tracking & Physical Activity MonitoringCHI
DisplayFab: The State of the Art and a Roadmap in the Personal Fabrication of Free-Form Displays Using Active Materials and Additive Manufacturing.Over recent years, there has been significant research within HCI towards free-form physical interactive devices. However, such devices are not straightforward to design, produce and deploy on demand. Traditional development revolves around iterative prototyping through component-based assembly, limiting device structure and implementation. Material-centric personal display fabrication (DisplayFab) opens the possibility of decentralised, configurable production by low-skill makers. Currently, DisplayFab is severely limited by its embryonic stage of development, the complexity of involved processes and materials, and the challenges around designing interactive structures. We present a development framework to provide a path for future research. DisplayFab has been developed by identifying 4 key breakpoints in the existing “Personal Fabrication” framework: Material and Deposition, Conception and Software, Feedback and Interactivity and Responsible Innovation. We use these breakpoints to form a targeted literature review of relevant work. Doing this we identify 30 challenges that act as roadmap for future research in DisplayFab.2024OHOllie Hanton et al.University of BathShape-Changing Interfaces & Soft Robotic MaterialsDesktop 3D Printing & Personal FabricationLaser Cutting & Digital FabricationCHI
Challenges and Opportunities for the Design of Inclusive Digital Mental Health Tools: Understanding Culturally Diverse Young People's ExperiencesMental health issues affect approximately 13% of people aged 10-24 years old worldwide. In Western countries (e.g. USA, UK, Australia), mental health issues are particularly prominent in Culturally and Linguistically Diverse (CALD) individuals, yet they are disproportionately affected in relation to service provision. Despite demand, there is a significant lack of literature explicitly exploring the design of digital mental health tools for CALD populations. Our study engaged five professionals working in CALD mental health, to gain insights into challenges for service access and provision, and then engaged 41 CALD young people to explore their experiences. We contribute a set of unique insights into the barriers that CALD young people face when seeking help, and their needs for future digital mental health tools. We also provide design recommendations for future researchers on how they might better support the inclusion of CALD communities in the design of digital health tools.2024ESEwan Soubutts et al.University College LondonCognitive Impairment & Neurodiversity (Autism, ADHD, Dyslexia)Universal & Inclusive DesignMental Health Apps & Online Support CommunitiesCHI
Watch This! Observational Learning in VR Promotes Better Far Transfer than Active Learning for a Fine Psychomotor TaskVirtual Reality (VR) holds great potential for psychomotor training, with existing applications using almost exclusively a `learning-by-doing' active learning approach, despite the possible benefits of incorporating observational learning. We compared active learning (n=26) with different variations of observational learning in VR for a manual assembly task. For observational learning, we considered three levels of visual similarity between the demonstrator avatar and the user, dissimilar (n=25), minimally similar (n=26), or a self-avatar (n=25), as similarity has been shown to improve learning. Our results suggest observational learning can be effective in VR when combined with `hands-on' practice and can lead to better far skill transfer to real-world contexts that differ from the training context. Furthermore, we found self-similarity in observational learning can be counterproductive when focusing on a manual task, and skills decay quickly without further training. We discuss these findings and derive design recommendations for future VR training.2024IFIsabel Sophie Fitton et al.University of BathFull-Body Interaction & Embodied InputImmersion & Presence ResearchSTEM Education & Science CommunicationCHI
Co-creating a Transdisciplinary Map of Technology-mediated Harms, Risks and Vulnerabilities: Challenges, Ambivalences and OpportunitiesThe phase ``online harms’’ has emerged in recent years out of a growing political willingness to address the ethical and social issues associated with the use of the internet and digital technology at large. The broad landscape that surrounds online harms gathers a multitude of disciplinary, sectoral and organizational efforts while raising myriad challenges and opportunities for the crossing entrenched boundaries. In this paper we draw lessons from a journey of co-creating a transdisciplinary knowledge infrastructure within a large research initiative animated by the online harms agenda. We begin with a reflection of the implications of mapping, taxonomizing and constructing knowledge infrastructures and a brief review of how online harm and adjacent themes have been theorized and classified in the literature to date. Grounded on our own experience of co-creating a map of online harms, we then argue that the map, and the process of mapping, performs three mutually constitutive functions, acting simultaneously as method, medium and provocation. We draw lessons from how an open-ended approach to mapping, despite not guaranteeing consensus, can foster productive debate and collaboration in ethically and politically fraught areas of research. We end with a call for CSCW research to surface and engage with the multiple temporalities, social lives and political sensibilities of knowledge infrastructures.2023AHAndrés Domínguez Hernández et al.Harm and VulnerabilityCSCW
Broadening Privacy and Surveillance: Eliciting Interconnected Values with a Scenarios Workbook on Smart Home Cameras We use a design workbook of speculative scenarios as a values elicitation activity with 14 participants. The workbook depicts use case scenarios with smart home camera technologies that involve surveillance and uneven power relations. The scenarios were initially designed by the researchers to explore scenarios of privacy and surveillance within three social relationships involving “primary” and “non-primary” users: Parents-Children, Landlords-Tenants, and Residents-Domestic Workers. When the scenarios were utilized as part of a values elicitation activity with participants, we found that they reflected on a broader set of interconnected social values beyond privacy and surveillance, including autonomy and agency, physical safety, property rights, trust and accountability, and fairness. The paper suggests that future research about ethical issues in smart homes should conceptualize privacy as interconnected with a broader set of social values (which can align or be in tension with privacy), and reflects on considerations for doing research with non-primary users.2023RWRichmond Y. Wong et al.Privacy by Design & User ControlPrivacy Perception & Decision-MakingSmart Home Privacy & SecurityDIS
Dancing with the Avatars: Minimal Avatar Customisation Enhances Learning in a Psychomotor TaskVirtual environments can support psychomotor learning by allowing learners to observe instructor avatars. Instructor avatars that look like the learner hold promise in enhancing learning; however, it is unclear whether this works for psychomotor tasks and how similar avatars need to be. We investigated `minimal’ customisation of instructor avatars, approximating a learner’s appearance by matching only key visual features: gender, skin-tone, and hair colour. These avatars can be created easily and avoid problems of highly similar avatars. Using modern dancing as a skill to learn, we compared the effects of visually similar and dissimilar avatars, considering both learning on a screen (n=59) and in VR (n=38). Our results indicate that minimal avatar customisation leads to significantly more vivid visual imagery of the dance moves than dissimilar avatars. We analyse variables affecting interindividual differences, discuss the results in relation to theory, and derive design implications for psychomotor training in virtual environments.2023IFIsabel Sophie Fitton et al.University of BathImmersion & Presence ResearchIdentity & Avatars in XRDance & Body Movement ComputingCHI