Exploring the Role of Interactive Technology to Enrich Surfing Surfing is not just a sport; it is a playful water activity rich in culture. Prior interaction design work to support surfers has mostly focused on improving performance; yet emphasizing performance misses the experiential value of play and enjoyment, which is under-investigated. To explore this opportunity, we engaged in a soma design process resulting in two prototypes, an actuating wearable top and an octopus-inspired soft robot, aiming to facilitate a playful experience to enrich surfing. We conducted an exploratory study with eight surfers in a swimming pool (acknowledging safety but also limitations). Through thematic analysis of interviews, we found three themes that supported the idea that the design features of the prototypes have the potential to enrich surfing. Moreover, adopting a postphenomenological lens, we investigate the Human-Technology-Water relations to understand the role of interactive technology during surfing and propose five design strategies that researchers can consider to develop future designs for the experiential aspects of surfing.2025MMMaria F. Montoya et al.Shape-Changing Interfaces & Soft Robotic MaterialsInteractive Narrative & Immersive StorytellingDIS
LuciEntry: Towards Understanding the Design of Lucid Dream InductionLucid dreaming, a state in which people become aware that they are dreaming, is known for its many mental and physical health benefits. However, most lucid dream induction techniques, such as reality testing, require significant time and effort to master, creating a barrier for people seeking these experiences. We designed \projectName, a portable interactive prototype aimed at helping people induce lucid dreaming through well-timed visual and auditory cues. We conducted a lab and a field study to understand \projectName{}'s user experience. The interview data allowed us to identify three themes. Building on these findings and our design practice, we derived seven considerations to guide the design of future lucid dream systems. Ultimately, this work aims to inspire further research into interactive technologies for altered states of consciousness.2025PWPo-Yao (Cosmos) Wang et al.Mental Health Apps & Online Support CommunitiesDIS
“My Happiness Makes You Smile”: Beginning to Understand Telepathic Superpower Design Via Brain-Muscle Interfaces Designing superpowers in Human-Computer Interaction (HCI), often inspired by science fiction, has garnered increased attention. However, it is important to ask whether such superpower designs might have inherent negative side effects, especially considering that technological advances allow going beyond short demos to integrate these superpowers into everyday life. To understand the positive and negative side effects of superpower design, we created "EmoPals" and studied it in everyday life. EmoPals is a novel system inspired by telepathy, where one user's emotions are detected through a brain-computer interface and replicated on the other user's face through electrical muscle stimulation, therefore one user's happiness makes the other smile and vice versa. A 5-day field study with 12 participants suggests that EmoPals can strengthen emotional connections and facilitate empathy, however, it also highlights the negative side effects of amplifying negative emotions and social discomfort. We propose five design recommendations for designing superpowers that account for negative side effects. Ultimately, we aim to deepen our understanding of superpower design for everyday life.2025SLSiyi Liu et al.Electrical Muscle Stimulation (EMS)Brain-Computer Interface (BCI) & NeurofeedbackDIS
Vision-Based Multimodal Interfaces: A Survey and Taxonomy for Enhanced Context-Aware System DesignThe recent surge in artificial intelligence, particularly in multimodal processing technology, has advanced human-computer interaction, by altering how intelligent systems perceive, understand, and respond to contextual information (i.e., context awareness). Despite such advancements, there is a significant gap in comprehensive reviews examining these advances, especially from a multimodal data perspective, which is crucial for refining system design. This paper addresses a key aspect of this gap by conducting a systematic survey of data modality-driven Vision-based Multimodal Interfaces (VMIs). VMIs are essential for integrating multimodal data, enabling more precise interpretation of user intentions and complex interactions across physical and digital environments. Unlike previous task- or scenario-driven surveys, this study highlights the critical role of the visual modality in processing contextual information and facilitating multimodal interaction. Adopting a design framework moving from the whole to the details and back, it classifies VMIs across dimensions, providing insights for developing effective, context-aware systems.2025YHYongquan 'Owen' Hu et al.University of New South WalesContext-Aware ComputingUbiquitous ComputingCHI
Shared Bodily Fusion: Leveraging Inter-Body Electrical Muscle Stimulation for Social PlayTraditional games like "Tag" rely on shared control via inter-body interactions (IBIs) – touching, pushing, and pulling – that foster emotional and social connection. Digital games largely limit IBIs, with players using their bodies as input to control virtual avatars instead. Our “Shared Bodily Fusion” approach addresses this by fusing players' bodies through a mediating computer, creating a shared input and output system. We demonstrate this approach with "Hidden Touch", a game where a novel social electrical muscle stimulation system transforms touch (input) into muscle actuations (output), facilitating IBIs. Through a study (n=27), we identified three player experience themes. Informed by these findings and our design process, we mapped their trajectories across our three experiential spaces – threshold, tolerance, and precision – which collectively form our design framework. This framework facilitates the creation of future digital games where IBIs are intrinsic, ultimately promoting the many benefits of social play.2024RPRakesh Patibanda et al.Electrical Muscle Stimulation (EMS)Serious & Functional GamesMultiplayer & Social GamesDIS
Exploring an Extended Reality Floatation Tank Experience to Reduce the Fear of Being in WaterPeople with a fear of being in water rarely engage in water activities and hence miss out on the associated health benefits. Prior research suggested virtual exposure to treat fears. However, when it comes to a fear of being in water, virtual water might not capture water’s immersive qualities, while real water can pose safety risks. We propose extended reality to combine both advantages: We conducted a study (N=12) where participants with a fear of being in water interacted with playful water-inspired virtual reality worlds while floating inside a floatation tank. Our findings, supported quantitatively by heart rate variability and qualitatively by interviews, suggest that playful extended reality could mitigate fear responses in an entertaining way. We also present insights for the design of future systems that aim to help people with a fear of being in water and other phobias by using the best of the virtual and physical worlds.2024MMMaria F. Montoya et al.Monash UniversityImmersion & Presence ResearchSTEM Education & Science CommunicationCHI
GustosonicSense: Towards understanding the design of playful gustosonic eating experiencesThe pleasure that often comes with eating can be further enhanced with intelligent technology, as the field of human-food interaction suggests. However, knowledge on how to design such pleasure-supporting eating systems is limited. To begin filling this knowledge gap, we designed “GustosonicSense”, a novel gustosonic eating system that utilizes wireless earbuds for sensing different eating and drinking actions with a machine learning algorithm and trigger playful sounds as a way to facilitate pleasurable eating experiences. We present the findings from our design and a study that revealed how we can support the "stimulation", "hedonism", and "reflexivity" for playful human-food interactions. Ultimately, with our work, we aim to support interaction designers in facilitating playful experiences with food.2024YWYan Wang et al.Monash University, Monash UniversityFood Culture & Food InteractionCHI
Grand challenges in WaterHCIRecent combinations of interactive technology, humans, and water have resulted in “WaterHCI”. WaterHCI design seeks to complement the many benefits of engagement with the aquatic domain, by offering, for example, augmented reality systems for snorkelers, virtual reality in floatation tanks, underwater musical instruments for artists, robotic systems for divers, and wearables for swimmers. We conducted a workshop in which WaterHCI experts articulated the field’s grand challenges, aiming to contribute towards a systematic WaterHCI research agenda and ultimately advance the field.2024FMFlorian Floyd Mueller et al.Monash UniversityDigital Art Installations & Interactive PerformanceDance & Body Movement ComputingCHI
From Plating to Tasting: Towards Understanding the Choreography of Computational Food The emerging concept of “computational food” focusing on the material affordances when designing food interactions is gaining traction in Human-Food Interaction (HFI). However, prior HFI research has not yet substantively investigated the dynamic nature of computational food from its creation to consumption, limiting our understanding of the complex interactions among creators, computational food, and consumers. In response, our paper shifts the perspective towards the dynamics of computational food interactions through a study in cooperation with chefs and gastronomists. Utilizing “Dancing Delicacies” as a research artifact – a system that facilitates dynamic dining trajectories – we adopted the concept of “choreography” to unravel the experiential dynamics of computational food. Our study resulted in six themes concerning computational food experiences and detailed four design implications central to culinary choreography. Our work aspires to leverage the choreographic potential of computational food design, paving the way for future HFI innovations.2024JDPatrick Olivier et al.Department of Human-Centred Computing, Monash UniversityMakerspace CultureFood Culture & Food InteractionCHI
Logic Bonbon: Exploring Food as Computational ArtifactIn recognition of food’s significant experiential pleasures, culinary practitioners and designers are increasingly exploring novel combinations of computing technologies and food. However, despite much creative endeavors, proposals and prototypes have so far largely maintained a traditional divide, treating food and technology as separate entities. In contrast, we present a “Research through Design” exploration of the notion of food as computational artifact: wherein food itself is the material of computation. We describe the Logic Bonbon, a dessert that can hydrodynamically regulate its flavor via a fluidic logic system. Through a study of experiencing the Logic Bonbon and reflection on our design practice, we offer a provisional account of how food as computational artifact can mediate new interactions through a novel approach to food-computation integration, that promotes an enriched future of Human-Food Interaction.2022JDJialin Deng et al.Monash UniversityUbiquitous ComputingFood Culture & Food InteractionCHI
Data as Delight: Eating dataThe HCI community has a rich history of finding new ways to engage people with data beyond the screen. With our work, we aim to expand the scope of how interaction design can engage people, arguing that “eating data” has the potential to allow people to experience “data as delight”. With reference to prior work and our design research findings, we discuss the advantages and the challenges of this approach to integrating data and food. We then identify four themes to guide the design of engagements with data through food: food form, food commensality, food ephemerality, and emotional response to food. Within these design themes, we articulate twelve insights for interaction designers to use when working on serving data as delight.2021FMFlorian Floyd Mueller et al.Monash UniversityFood Culture & Food InteractionCHI
What can HCI learn from sexual consent? A Feminist Process of Embodied Consent for Interactions with Emerging TechnologiesSexual consent has undergone a transformation toward an “enthusiastic” feminist model that emphasizes consent as an ongoing and voluntary process of negotiation and affirmation. This paper considers how such a model can advance understandings of consent in HCI research and design in relation to embodied interactions with emerging technologies that also occur outside of sexual interactions. We apply the popular “FRIES” model of sexual consent (Freely given, Reversible, Informed, Enthusiastic and Specific) to three areas of embodied interaction: 1) bodily-play interactions, 2) persuasive interactions with smart technologies, and 3) intimate interactions with anthropomorphized devices. Based on erotic play practices, we contribute a “TEASE” process guideline (Traffic lights, Establish ongoing dialogue, Aftercare, Safewords, and Explicate soft/hard limits) to advance consensual practice in HCI and develop implementation scenarios.2021YSYolande Strengers et al.Monash UniversityTechnology Ethics & Critical HCIParticipatory DesignInteractive Narrative & Immersive StorytellingCHI
Limited control over the body as intriguing play design resourceInterest in combining interactive play and the human body, using “bodily play” systems, is increasing. While these systems primarily prioritize a player’s control over their bodily actions, we see intriguing possibilities in the pursuit of “limited control over the body” as an intriguing design resource for bodily play systems. In this paper, we use three of our bodily play systems to illustrate how designers can engage with limited control over the body by varying the player’s degree of indirect control (for instance, via other bodily activity and external triggers). We also propose four strategies for employing limited control over the body: Exploration, Reflection, Learning and Embracement. We hope our own work and the strategies developed from it will assist designers to employ limited control over the body, ultimately helping people benefit from engaging their bodies through play.2021FMFlorian Floyd Mueller et al.Monash UniversityFull-Body Interaction & Embodied InputCHI
Designing Drone Chi: Unpacking the Thinking and Making of Somaesthetic Human-Drone InteractionDrone Chi is a Tai Chi inspired human-drone interaction experience. As a design research project, situated within somaesthetic interaction design, where a central topic is cultivating bodily and sensory appreciation to improve one’s quality of life. Drone Chi investigates the potential of autonomous micro-quadcopters as a design material for somaesthetic HCI. Through a quasi-chronological account of the design process, this pictorial articulates how the sensory experiences of Tai Chi were integrated into Drone Chi. Taking a slow and open-ended design research approach, we iteratively developed the project through somaesthetic, product design and engineering perspectives and drew heavily on design analogies and imagery for inspiration. This elevated the influence of the soma amongst narrow engineering parameters and usability requirements. This pictorial serves as a reflective resource for designers who are experimenting with merging their native discipline with someasthetic interaction design.2020JDJoseph La Delfa et al.Full-Body Interaction & Embodied InputDrone Interaction & ControlDIS
InsideOut: Towards an Understanding of Designing Playful Experiences with Imaging CapsulesImaging capsules are ingestible sensors that capture the video of one's gastrointestinal tract for medical diagnosis. We believe that the capsule’s experiential perspective is often overlooked by associated medical applications. This work explores the design of this experiential perspective through combining imaging capsules with digital play. We designed a playful wearable system called "InsideOut", where users play with the real-time video of their gastrointestinal tract captured by an imaging capsule. Based on an in-the-wild study, we derived four themes articulating the play experiences and discussed key design implications to guide future playful designs using imaging capsules. Our research highlights the opportunity of using medical imaging technologies to enable intriguing bodily play experiences. Furthermore, such experiences can deepen the players’ engagement with and understanding of their bodies, ultimately contributing to a more playful and humanized health care agenda.2020ZLZhuying Li et al.Vibrotactile Feedback & Skin StimulationSerious & Functional GamesDIS
WeScream!: Toward understanding the design of playful social gustosonic experiences with ice creamPrior psychology studies have shown that eating ice cream increases happiness, while human–computer interaction work has shown that interactive technology can enrich the eating experience. We explore the opportunity to combine these two through WeScream!, a playful social gustosonic system we designed—social gustosonic referring to the link between the acts of eating and listening as part of a social multisensory experience. WeScream! consists of two interdependent ice-cream cones that allow users to interact with musical sounds generated through the act of eating ice cream together. We report on an in-the-wild study that highlights how our system facilitated a “hard fun” experience through eating together, increased participants’ awareness of relatedness, and drew shared attention to the ice cream’s taste via increased face-to-face interaction. Drawing on these study insights, we also present three design tactics to guide designers in designing future social gustosonic experiences. Ultimately, we aim to contribute to a playful future of social eating experiences, supporting people in enjoying eating together.2020YWYan Wang et al.Universal & Inclusive DesignInteractive Narrative & Immersive StorytellingFood Culture & Food InteractionDIS
Neo-Noumena: Augmenting Emotion CommunicationThe subjective experience of emotion is notoriously difficult to interpersonally communicate. We believe that technology can challenge this notion through the design of neuroresponsive systems for interpersonal communication. We explore this through "Neo-Noumena", a communicative neuroresponsive system that uses brain-computer interfacing and artificial intelligence to read one's emotional states and dynamically represent them to others in mixed reality through two head-mounted displays. In our study five participant pairs were given Neo-Noumena for three days, using the system freely. Measures of emotional competence demonstrated a statistically significant increase in participants' ability to interpersonally regulate emotions. Furthermore, participant interviews revealed themes regarding Spatiotemporal Actualization, Objective Representation, and Preternatural Transmission. We also suggest design strategies for future augmented emotion communication systems. We intend that work gives guidance towards a future in which our ability to interpersonally communicate emotion is augmented beyond traditional experience.2020NSNathan Semertzidis et al.Monash UniversityBrain-Computer Interface (BCI) & NeurofeedbackMixed Reality WorkspacesCHI
Drone Chi: Somaesthetic Human-Drone InteractionSomaesthetics — motivated by improving life quality via appreciation for bodily and sensory experiences — is increasingly influencing HCI designs. Investigating the potential of drones as a material for somaesthetic HCI, we designed Drone Chi: a Tai Chi-inspired close-range human-drone interaction experience. The design process for Drone Chi has been informed by the soma design approach and the Somaesthetic Appreciation concept from HCI literature. The artifact expands somaesthetic HCI by exemplifying dynamic and intimate somaesthetic interactions with a robotic design material, and body movements in expansive 3D space. To characterize the Drone Chi experience, we conducted an empirical study with 32 participants. Analysis of participant accounts revealed 4 themes that articulate different aspects of the experience: Looping Mental States, Environment, Agency vs. Control, and Physical Narratives. From these accounts and our craft knowledge, we derive 5 design implications to guide the development of movement-based close-range drone interactions.2020JDJoseph Delfa et al.Monash UniversityFull-Body Interaction & Embodied InputDrone Interaction & ControlDance & Body Movement ComputingCHI
Next Steps for Human-Computer IntegrationHuman-Computer Integration (HInt) is an emerging paradigm in which computational and human systems are closely interwoven. Integrating computers with the human body is not new. however, we believe that with rapid technological advancements, increasing real-world deployments, and growing ethical and societal implications, it is critical to identify an agenda for future research. We present a set of challenges for HInt research, formulated over the course of a five-day workshop consisting of 29 experts who have designed, deployed and studied HInt systems. This agenda aims to guide researchers in a structured way towards a more coordinated and conscientious future of human-computer integration.2020FMFlorian Floyd Mueller et al.Monash UniversityBrain-Computer Interface (BCI) & NeurofeedbackTechnology Ethics & Critical HCIUser Research Methods (Interviews, Surveys, Observation)CHI
Introducing Peripheral Awareness as a Neurological State for Human-computer IntegrationIn this work we introduce peripheral awareness as a neurological state for real-time human-computer integration, where the human is assisted by a computer to interact with the world. Changes to the field of view in peripheral awareness have been linked with quality of human performance. This instinctive narrowing of vision that occurs as a threat is perceived has implications in activities that benefit from the user having a wide field of view, such as cycling to navigate the environment. We present "Ena", a novel EEG-eBike system that draws from the user's neural activity to determine when the user is in a state of peripheral awareness to regulate engine support. A study with 20 participants revealed various themes and tactics suggesting that peripheral awareness as a neurological state is viable to align human-machine integration with internal bodily processes. Ena suggests that our work facilitates a safe and enjoyable human-computer integration experience.2020JAJosh Andres et al.Monash University & IBM ResearchMicromobility (E-bike, E-scooter) InteractionHand Gesture RecognitionBrain-Computer Interface (BCI) & NeurofeedbackCHI