Towards Understanding the Design of Mixed Reality Systems to Enrich the Beverage ExperienceDrinking is an inherently multisensory activity, yet the potential of immersive technology to dynamically shape flavor experiences remains underexplored in Human-Food Interaction (HFI) research. We introduce “XTea”, an adaptive beverage cup-based system that integrates large language models to translate natural language input into modifications of a parameterized immersive environment experienced through a headset when drinking bubble tea. Through a study with 12 bubble tea enthusiasts, we derived themes that demonstrate how “XTea” can enrich sensory engagement, support personalized and agentic experiences, and foster social qualities of drinking, pointing toward new explorations for multisensory HFI design. We also present four design strategies for multisensory beverage experiences. Ultimately, we aim to contribute to the advancement of HFI research on how multisensory interaction design can enrich flavor perception and engagement.2026YZYuchen Zheng et al.Monash UniversityMultisensory Fusion ExperienceHuman-Nature Relationships (More-than-Human Design)Generative AI (Text, Image, Music, Video)CHI
Grand Challenges around Designing Computers’ Control Over Our BodiesAdvances in emerging technologies, such as on-body mechanical actuators and electrical muscle stimulation, have allowed computers to take control over our bodies. This presents opportunities as well as challenges, raising fundamental questions about agency and the role of our body when interacting with technology. To advance this research field as a whole, we brought together expert perspectives in a week-long seminar to articulate the grand challenges that should be tackled when it comes to the design of computers’ control over our bodies. These grand challenges span technical, design, user, and ethical aspects. By articulating these grand challenges, we aim to begin initiating a research agenda that positions bodily control not only as a technical feature but as a central, experiential, and ethical concern for future human–computer interaction endeavors.2026FMFlorian 'Floyd' Mueller et al.Monash UniversityElectrical Muscle Stimulation (EMS)Brain-Computer Interface (BCI) & NeurofeedbackEmpathy & Emotional DesignCHI
GastroConcerto: Towards Designing Dining–Sound Pairings to Support Culinary CreativitySounds play a crucial role in shaping dining experiences. Recently, designers have increasingly integrated them into interfaces that connect diners with food. Yet, little is known about how sounds can function as culinary materials to enrich chefs' creativity, particularly in creating meaningful auditory interactions that resonate with their culinary creations. Hence, we present GastroConcerto, an auditory dining system that combines a magnetic contact microphone equipped under the plate with a companion mobile application. This system delves into the interactive space between diners and their food to introduce a novel interactive mechanism of dining–sound pairing, enabling chefs to design specific auditory interactions that respond to diners' individual interactions with the dining containers. Through a within-subjects study and field deployment, we examined how GastroConcerto enriches chefs' creative practices in crafting "sonic dish" experiences. Ultimately, our goal is to shift the ownership of auditory interaction design from interface designers to chefs, thereby supporting their culinary creativity.2026HWHongyue Wang et al.Monash UniversityOlfactory Display & Smell InteractionMultisensory Fusion ExperienceEmpathy & Emotional DesignCHI
LuciEntry: Towards Understanding the Design of Lucid Dream InductionLucid dreaming, a state in which people become aware that they are dreaming, is known for its many mental and physical health benefits. However, most lucid dream induction techniques, such as reality testing, require significant time and effort to master, creating a barrier for people seeking these experiences. We designed \projectName, a portable interactive prototype aimed at helping people induce lucid dreaming through well-timed visual and auditory cues. We conducted a lab and a field study to understand \projectName{}'s user experience. The interview data allowed us to identify three themes. Building on these findings and our design practice, we derived seven considerations to guide the design of future lucid dream systems. Ultimately, this work aims to inspire further research into interactive technologies for altered states of consciousness.2025PWPo-Yao (Cosmos) Wang et al.Mental Health Apps & Online Support CommunitiesDIS
“My Happiness Makes You Smile”: Beginning to Understand Telepathic Superpower Design Via Brain-Muscle Interfaces Designing superpowers in Human-Computer Interaction (HCI), often inspired by science fiction, has garnered increased attention. However, it is important to ask whether such superpower designs might have inherent negative side effects, especially considering that technological advances allow going beyond short demos to integrate these superpowers into everyday life. To understand the positive and negative side effects of superpower design, we created "EmoPals" and studied it in everyday life. EmoPals is a novel system inspired by telepathy, where one user's emotions are detected through a brain-computer interface and replicated on the other user's face through electrical muscle stimulation, therefore one user's happiness makes the other smile and vice versa. A 5-day field study with 12 participants suggests that EmoPals can strengthen emotional connections and facilitate empathy, however, it also highlights the negative side effects of amplifying negative emotions and social discomfort. We propose five design recommendations for designing superpowers that account for negative side effects. Ultimately, we aim to deepen our understanding of superpower design for everyday life.2025SLSiyi Liu et al.Electrical Muscle Stimulation (EMS)Brain-Computer Interface (BCI) & NeurofeedbackDIS
Towards Understanding Interactive Sonic Gastronomy with Chefs and DinersWith advancements in interactive technologies, research in human-food interaction (HFI) has begun to employ interactive sound to enrich the dining experience. However, chefs' creative use of this sonic interactivity as a new "ingredient" in their culinary practices remains underexplored. In response, we conducted an empirical study with six pairs of chefs and diners utilizing SoniCream, an ice cream cone that plays digital sounds while consuming. Through exploration, creation, collaboration, and reflection, we identified four themes concerning culinary creativity, dining experience, interactive sonic gastronomy deployment, and chef-diner interplay. Building on the discussions at the intersection of these themes, we derived four design implications for creating interactive systems that could support chefs' culinary creativity, thereby enriching dining experiences. Ultimately, our work aims to help interaction designers fully incorporate chefs' perspectives into HFI research.2025HWHongyue Wang et al.Monash University, Exertion Games Lab, Human-Centred ComputingHaptic WearablesFood Culture & Food InteractionCHI
Vision-Based Multimodal Interfaces: A Survey and Taxonomy for Enhanced Context-Aware System DesignThe recent surge in artificial intelligence, particularly in multimodal processing technology, has advanced human-computer interaction, by altering how intelligent systems perceive, understand, and respond to contextual information (i.e., context awareness). Despite such advancements, there is a significant gap in comprehensive reviews examining these advances, especially from a multimodal data perspective, which is crucial for refining system design. This paper addresses a key aspect of this gap by conducting a systematic survey of data modality-driven Vision-based Multimodal Interfaces (VMIs). VMIs are essential for integrating multimodal data, enabling more precise interpretation of user intentions and complex interactions across physical and digital environments. Unlike previous task- or scenario-driven surveys, this study highlights the critical role of the visual modality in processing contextual information and facilitating multimodal interaction. Adopting a design framework moving from the whole to the details and back, it classifies VMIs across dimensions, providing insights for developing effective, context-aware systems.2025YHYongquan 'Owen' Hu et al.University of New South WalesContext-Aware ComputingUbiquitous ComputingCHI
Shared Bodily Fusion: Leveraging Inter-Body Electrical Muscle Stimulation for Social PlayTraditional games like "Tag" rely on shared control via inter-body interactions (IBIs) – touching, pushing, and pulling – that foster emotional and social connection. Digital games largely limit IBIs, with players using their bodies as input to control virtual avatars instead. Our “Shared Bodily Fusion” approach addresses this by fusing players' bodies through a mediating computer, creating a shared input and output system. We demonstrate this approach with "Hidden Touch", a game where a novel social electrical muscle stimulation system transforms touch (input) into muscle actuations (output), facilitating IBIs. Through a study (n=27), we identified three player experience themes. Informed by these findings and our design process, we mapped their trajectories across our three experiential spaces – threshold, tolerance, and precision – which collectively form our design framework. This framework facilitates the creation of future digital games where IBIs are intrinsic, ultimately promoting the many benefits of social play.2024RPRakesh Patibanda et al.Electrical Muscle Stimulation (EMS)Serious & Functional GamesMultiplayer & Social GamesDIS
Exploring an Extended Reality Floatation Tank Experience to Reduce the Fear of Being in WaterPeople with a fear of being in water rarely engage in water activities and hence miss out on the associated health benefits. Prior research suggested virtual exposure to treat fears. However, when it comes to a fear of being in water, virtual water might not capture water’s immersive qualities, while real water can pose safety risks. We propose extended reality to combine both advantages: We conducted a study (N=12) where participants with a fear of being in water interacted with playful water-inspired virtual reality worlds while floating inside a floatation tank. Our findings, supported quantitatively by heart rate variability and qualitatively by interviews, suggest that playful extended reality could mitigate fear responses in an entertaining way. We also present insights for the design of future systems that aim to help people with a fear of being in water and other phobias by using the best of the virtual and physical worlds.2024MMMaria F. Montoya et al.Monash UniversityImmersion & Presence ResearchSTEM Education & Science CommunicationCHI
Grand challenges in WaterHCIRecent combinations of interactive technology, humans, and water have resulted in “WaterHCI”. WaterHCI design seeks to complement the many benefits of engagement with the aquatic domain, by offering, for example, augmented reality systems for snorkelers, virtual reality in floatation tanks, underwater musical instruments for artists, robotic systems for divers, and wearables for swimmers. We conducted a workshop in which WaterHCI experts articulated the field’s grand challenges, aiming to contribute towards a systematic WaterHCI research agenda and ultimately advance the field.2024FMFlorian Floyd Mueller et al.Monash UniversityDigital Art Installations & Interactive PerformanceDance & Body Movement ComputingCHI
PsiNet: Toward Understanding the Design of Brain-to-Brain Interfaces for Augmenting Inter-Brain SynchronyUnderlying humanity’s social abilities is the brain’s capacity to interpersonally synchronize. Experimental, lab-based neuropsychological studies have demonstrated that inter-brain synchrony can be technologically mediated. However, knowledge in deploying these technologies in-the-wild and studying their user experience, an area HCI excels in, is lacking. With advances in mobile brain sensing and stimulation, we identify an opportunity for HCI to investigate the in-the-wild augmentation of inter-brain synchrony. We designed “PsiNet,” the first wearable brain-to-brain system aimed at augmenting inter-brain synchrony in-the-wild. Participant interviews illustrated three themes that describe the user experience of modulated inter-brain synchrony: hyper-awareness; relational interaction; and the dissolution of self. We contribute these three themes to assist HCI theorists’ discussions of inter-brain synchrony experiences. We also present three practical design tactics for HCI practitioners designing inter-brain synchrony, and hope that our work guides a HCI future of brain-to-brain experiences which fosters human connection.2024NSNathan Semertzidis et al.Monash UniversityBrain-Computer Interface (BCI) & NeurofeedbackUbiquitous ComputingCHI
Grand Challenges in SportsHCIThe field of Sports Human-Computer Interaction (SportsHCI) investigates interaction design to support a physically active human being. Despite growing interest and dissemination of SportsHCI literature over the past years, many publications still focus on solving specific problems in a given sport. We believe in the benefit of generating fundamental knowledge for SportsHCI more broadly to advance the field as a whole. To achieve this, we aim to identify the grand challenges in SportsHCI, which can help researchers and practitioners in developing a future research agenda. Hence, this paper presents a set of grand challenges identified in a five-day workshop with 22 experts who have previously researched, designed, and deployed SportsHCI systems. Addressing these challenges will drive transformative advancements in SportsHCI, fostering better athlete performance, athlete-coach relationships, spectator engagement, but also immersive experiences for recreational sports or exercise motivation, and ultimately, improve human well-being.2024DEDon Samitha Elvitigala et al.Monash UniversityGame UX & Player BehaviorSerious & Functional GamesMental Health Apps & Online Support CommunitiesCHI
RadarFoot: Fine-grain Ground Surface Context Awareness for Smart ShoesEveryday, billions of people use footwear for walking, running, or exercise. Of emerging interest are ``smart footwear'', which help users track gait, count steps or even analyse performance. However, such nascent footwear lack fine-grain ground surface context awareness, which could allow them to adapt to the conditions and create usable functions and experiences. Hence, this research aims to recognize the walking surface using a radar sensor embedded in a shoe, enabling ground context-awareness. Using data collected from 23 participants from an in-the-wild setting, we developed several classification models. We show that our model can detect five common terrain types with an accuracy of 80.0\% and further ten terrain types with an accuracy of 66.3\%, while moving. Importantly, it can detect the gait motion types such as `walking', `stepping up', `stepping down', `still', with an accuracy of 90\%. Finally, we present potential use cases and insights for future work based on such ground-aware smart shoes.2023DEDon Samitha Elvitigala et al.Biosensors & Physiological MonitoringContext-Aware ComputingUIST