Domestic Cultures of Plant Care: A Moss Terrarium ProbeHouseplants are increasingly being used as part of interactive systems that aim to foster pro-environmental concern and awareness of more-than-human life. Yet such interventions rely on conflicting and untested assumptions about how people relate to houseplants. We therefore studied domestic plant care in 11 purposefully sampled households, applying a sensor-equipped moss terrarium as a living ‘thing ethnography’ probe, supplemented with semi-structured interviews. We find that social and intergenerational cultures of plant care inform people's individual concern and accountability through constituents and mechanisms like gift-giving, signalling, knowledge transfer, or joint practical care. We identify five domestic cultures of plant care in our sample, each of which frames plants differently and leads to different practical approaches to plant care. We propose design considerations that emphasise enculturation and shared care over individual behaviour change and reframe houseplants from decorative objects into living household members.2025NBNirit Binyamini Ben-Meir et al.Sustainable HCIHuman-Nature Relationships (More-than-Human Design)DIS
Exclusion Rates among Disabled and Older Users of Virtual and Augmented RealityThis paper examines the levels of exclusion encountered by disabled and older users of consumer-level VR and AR technology and identifies methods formed by people with diverse access needs to circumvent encountered barriers to use. First, we estimate exclusion rates for a selection of nine immersive experiences of VR and AR, computed using population statistics data for the United Kingdom (UK). We then present an empirical lab-based study evaluating the usability of the same VR and AR experiences. The study involved 60 UK-based participants with varying access needs and the study results were used to calculate the empirical exclusion rates. Both the estimated and empirical exclusion rates display high levels of exclusion, which for the more complex experiences in the study reached 100%. However, multiple participants overcame usability barriers and completed experiences through provided assistance and self-initiated adaptations, suggesting that future VR and AR can become more inclusive if designed to counter these barriers.2025RERosella P. Galindo Esparza et al.Brunel University London, Brunel Design SchoolIdentity & Avatars in XRUniversal & Inclusive DesignCHI
Archaeological Gameworld Affordances: A Grounded Theory of How Players Interpret Environmental StorytellingEnvironmental storytelling is a design technique commonly used to convey narrative through assemblages of content in video games. To date there has been limited empirical work investigating how and on what basis players form interpretations about game environments. We report on a study in which participants (N=202) played a game about exploring a procedurally generated ruined village and were then surveyed on their interpretations. We draw on methods and theory from archaeology - a field that specialises in the interpretation of material remains - to support a grounded theory analysis of the survey responses, from which we form the theory of an archaeological gameworld mental model. Our study draws a novel link between affordance theory, archaeological knowledge production and game systems, and contributes new theoretical concepts that can be applied to procedurally generated and handcrafted methods in game design, narrative design and game preservation.2025FNFlorence Smith Nicholls et al.Queen Mary University of LondonRole-Playing & Narrative GamesInteractive Narrative & Immersive StorytellingCHI
“Actually I Can Count My Blessings”: User-Centered Design of an Application to Promote Gratitude Among Young AdultsRegular practice of gratitude has the potential to enhance psychological wellbeing and foster stronger social connections among young adults. However, there is a lack of research investigating user needs and expectations regarding gratitude-promoting applications. To address this gap, we employed a user-centered design approach to develop a mobile application that facilitates gratitude practice. Our formative study involved 20 participants who utilized an existing application, providing insights into their preferences for organizing expressions of gratitude and the significance of prompts for reflection and mood labeling after working hours. Building on these findings, we conducted a deployment study with 26 participants using our custom-designed application, which confirmed the positive impact of structured options to guide gratitude practice and highlighted the advantages of passive engagement with the application during busy periods. Our study contributes to the field by identifying key design considerations for promoting gratitude among young adults.2024ABAnanya Bhattacharjee et al.Session 3d: Teens in the Digital Age: Safety, Creativity, and Well-BeingCSCW
Using Incongruous Genres to Explore Music Making with AI Generated ContentDeep learning generative AI models trained on huge datasets are capable of producing complex and high quality music. However, there are few studies of how AI Generated Content (AIGC) is actually used or appropriated in creative practice. We present two first-person accounts by musician-researchers of explorations of an interactive generative AI system trained on Irish Folk music. The AI is intentionally used by musicians from incongruous genres of Punk and Glitch to explore questions of how the model is appropriated into creative practice and how it changes creative practice when used outside of its intended genre. Reflections on the first-person accounts highlight issues of control, ambiguity, trust, and filtering AIGC. The accounts also highlight the role of AI as an audience and critic and how the musicians’ practice changed in response to the AIGC. We suggest that our incongruous approach may help to foreground the creative work and frictions in human-AI creative practice.2024NBNick Bryan-Kinns et al.Generative AI (Text, Image, Music, Video)AI-Assisted Creative WritingMusic Composition & Sound Design ToolsC&C
Reflection Across AI-based Music CompositionReflection is fundamental to creative practice. However, the plurality of ways in which people reflect when using AI Generated Content (AIGC) is underexplored. This paper takes AI-based music composition as a case study to explore how artist-researcher composers reflected when integrating AIGC into their music composition process. The AI tools explored range from Markov Chains for music generation to Variational Auto-Encoders for modifying timbre. We used a novel method where our composers would pause and reflect back on screenshots of their composing after every hour, using this documentation to write first-person accounts showcasing their subjective viewpoints on their experience. We triangulate the first-person accounts with interviews and questionnaire measures to contribute descriptions on how the composers reflected. For example, we found that many composers reflect on future directions in which to take their music whilst curating AIGC. Our findings contribute to supporting future explorations on reflection in creative HCI contexts.2024CFCorey Ford et al.Generative AI (Text, Image, Music, Video)Music Composition & Sound Design ToolsCreative Collaboration & Feedback SystemsC&C
How does Juicy Game Feedback Motivate? Testing Curiosity, Competence, and Effectance'Juicy' or immediate abundant action feedback is widely held to make video games enjoyable and intrinsically motivating. Yet we do not know why it works: Which motives are mediating it? Which features afford it? In a pre-registered (n=1,699) online experiment, we tested three motives mapping prior practitioner discourse---effectance, competence, and curiosity---and connected design features. Using a dedicated action RPG and a 2x2+control design, we varied feedback amplification, success-dependence, and variability and recorded self-reported effectance, competence, curiosity, and enjoyment as well as free-choice playtime. Structural equation models show curiosity as the strongest enjoyment and only playtime predictor and support theorised competence pathways. Success dependence enhanced all motives, while amplification unexpectedly reduced them, possibly because the tested condition unintentionally impeded players' sense of agency. Our study evidences uncertain success affording curiosity as an underappreciated moment-to-moment engagement driver, directly supports competence-related theory, and suggests that prior juicy game feel guidance ties to legible action-outcome bindings and graded success as preconditions of positive 'low-level' user experience.2024DKDominic Kao et al.Purdue UniversityGame UX & Player BehaviorGamification DesignCHI
Not All the Same: Understanding and Informing Similarity Estimation in Tile-Based Video GamesSimilarity estimation is essential for many game AI applications, from the procedural generation of distinct assets to automated exploration with game-playing agents. While similarity metrics often substitute human evaluation, their alignment with our judgement is unclear. Consequently, the result of their application can fail human expectations, leading to e.g. unappreciated content or unbelievable agent behaviour. We alleviate this gap through a multi-factorial study of two tile-based games in two representations, where participants (N=456) judged the similarity of level triplets. Based on this data, we construct domain-specific perceptual spaces, encoding similarity-relevant attributes. We compare 12 metrics to these spaces and evaluate their approximation quality through several quantitative lenses. Moreover, we conduct a qualitative labelling study to identify the features underlying the human similarity judgement in this popular genre. Our findings inform the selection of existing metrics and highlight requirements for the design of new similarity metrics benefiting game development and research.2024SBSebastian Berns et al.Queen Mary University of LondonGame UX & Player BehaviorRole-Playing & Narrative GamesCHI
RoomDreaming: Generative-AI Approach to Facilitating Iterative, Preliminary Interior Design ExplorationInterior design aims to create aesthetically pleasing and functional environments within an architectural space. For a simple room, the preliminary design exploration currently takes multiple meetings and days of work for interior designers to incorporate homeowners' personal preferences through layout, furnishings, form, colors, and materials. We present RoomDreaming, a generative AI-based approach designed to facilitate preliminary interior design exploration. It empowers owners and designers to rapidly and efficiently iterate through a broad range of AI-generated, photo-realistic design alternatives, each uniquely tailored to fit actual space layouts and individual design preferences. We conducted a series of formative and summative studies with a total of 18 homeowners and 20 interior designers to help design, improve, and evaluate RoomDreaming. Owners reported that RoomDreaming effectively increased the breadth and depth of design exploration with higher efficiency and satisfaction. Designers reported that one hour of collaborative designing with RoomDreaming yielded results comparable to several days of traditional owner-designer meetings, plus days to weeks worth of designer work to develop and refine designs.2024SWShun-Yu Wang et al.National Taiwan UniversityGenerative AI (Text, Image, Music, Video)Customizable & Personalized ObjectsCHI
Thinking with Sound: Exploring the Experience of Listening to an Ultrasonic Art Installation Entanglement theories are well established in HCI discourse. These involve a commitment to view human experience in encounters with technology as relational and contingent, and research apparatuses as co-producers rather than passive observers of phenomena. In this paper, we argue that sound is the sensory modality best suited to the investigation of entanglements. Materialist theories of sound and listening guide both the design of a novel interactive sound installation and the methodological approach of a participant study exploring the experience of listening. We present a diffractive analysis whereby micro-phenomenological interview data is read with sonic theories, generating accounts that might otherwise remain mute: the temporal fluctuation and physical feeling of proximity in listener entanglements with sound, somatic intention setting, and plural interpretations of interactivity. Finally, we offer a series of provocations for HCI to embrace qualities of the sonic and consider epistemological positions grounded in other sense modalities.2024NRNicole Robson et al.Queen Mary University of LondonMid-Air Haptics (Ultrasonic)Digital Art Installations & Interactive PerformanceCHI
What the Sensor Knows: More-Than-Human Knowledge Co-Production in Wood CarvingFrom an engineering perspective, sensors provide measurements of phenomena in the world. Sensor data might be noisy, biased or otherwise subject to error, and these labels presuppose the existence of an objective ground truth the sensor is intended to approximate. This paper explores an alternative look at sensor signals as situated observations entangled with the systems they seek to measure, where meaning can be carried in qualitative particulars rather than quantitative and statistical analyses. Karen Barad’s `agential realism' \cite{baradMeetingUniverseHalfway2007} and Graham Harman’s `tool-being' \cite{harmanToolbeingHeideggerMetaphyics2002} inform our approach to understanding \emph{together with} the sensors, which become co-investigators and co-creators of the subsequent knowledge. We illustrate our collaborative effort with the sensor through a case study where we examine sensor signals from a device designed to query the experience of woodcarving by making the experience unfamiliar. We seek a qualitative approach to knowledge (co-)creation with sensor data from a more-than-human perspective.2023CNCharlotte Nordmoen et al.Augmentative & Alternative Communication (AAC)Digital Art Installations & Interactive PerformanceDIS
When Materials Meet Sound: Discovering the Meaning of Deformable Materials in Musical InteractionResearch on Digital Musical Instruments (DMIs) design highlights that materiality plays an important role in DMI design and musical interaction. However, DMI design research often focuses on technology-oriented factors, with less exploration of the meaning of materials in design practice. In this paper, we explore how DMI designers understand deformable sensor materials and how they use these as a resource for creative aesthetic design. Eleven DMI designers were invited to use a selection of deformable sensor materials to create prototype DMIs with them in a design activity. Three design approaches emerged, determined by how designers perceived and explored sensor materials. We discuss the potential of the methodology for exploring strongly entangled elements, such as material, gesture, and sound, in DMI design. The results contribute to the design practice for DMI designers and to further exploration of material-based design research in Human-Computer Interaction.2023JZJianing Zheng et al.Shape-Changing Interfaces & Soft Robotic MaterialsMusic Composition & Sound Design ToolsDIS
Towards a Reflection in Creative Experience QuestionnaireReflection is underexplored in Creativity Support Tool (CST) research, partly due to its ambiguous nature. We suggest that researchers could benefit from a measure of a CST's capacity to support reflection. To this end, we detail the first stages of development of the Reflection in Creative Experience Questionnaire (RiCE) – a lightweight questionnaire for differentiating between creative user experiences which exhibit more or less moments of reflection. We develop RiCE through i) an expert review of questionnaire items (n=10) and ii) an exploratory factor analysis (n=300) of the reviewed items. We also present a user study testing RiCE (n=58) across two time points (one week apart) with novel interfaces designed for creative writing and music making. Although we do not confirm validity, we identify four factors for RiCE which we suggest are interpretable in a conceptually meaningful way. Our formative studies contribute towards supporting future explorations on reflection with CSTs.2023CFCorey Ford et al.Queen Mary University of LondonMental Health Apps & Online Support CommunitiesCreative Collaboration & Feedback SystemsCHI
Negotiating Experience and Communicating Information Through Abstract MetaphorAn implicit assumption in metaphor use is that it requires grounding in a familiar concept, prominently seen in the popular Desktop Metaphor. In human-to-human communication, however, abstract metaphors, without such grounding, are often used with great success. To understand when and why metaphors work, we present a case study of metaphor use in voice teaching. Voice educators must teach about subjective, sensory experiences and rely on abstract metaphor to express information about unseen and intangible processes inside the body. We present a thematic analysis of metaphor use by 12 voice teachers. We found that metaphor works not because of strong grounding in the familiar, but because of its ambiguity and flexibility, allowing shared understanding between individual lived experiences. We summarise our findings in a model of metaphor-based communication. This model can be used as an analysis tool within the existing taxonomies of metaphor in user interaction for better understanding why metaphor works in HCI. It can also be used as a design resource for thinking about metaphor use and abstracting metaphor strategies from both novel and existing designs.2023CRCourtney N. Reed et al.Max Planck Institute for Informatics, Saarland Informatics Campus, Queen Mary University of LondonUser Research Methods (Interviews, Surveys, Observation)Interactive Narrative & Immersive StorytellingCHI
Identifying Engagement in Children’s Interaction whilst Composing Digital Music at HomeIdentifying points of engagement from a person's interaction with computers could be used to assess their experience and to adapt user interfaces in real-time. However, it is difficult to identify points of engagement unobtrusively; HCI studies typically use retrospective protocols or rely on cumbersome sensors for real-time analysis. We present a case study on how children compose digital music at home in which we remotely identify points of engagement from patterns of interaction with a musical interface. A mixed-methods approach is contributed in which video recordings of children's interactions whilst composing are labelled for engagement and linked to i) interaction logs from the interface to identify indicators of engagement in interaction, and ii) interview data gathered using a remote video-cued recall technique to understand the experiential qualities of engaging interactions directly from users. We conclude by speculating on how the suggested indicators of engagement might inform the design of adaptive music systems.2022CFCorey Ford et al.Music Composition & Sound Design ToolsCreative Collaboration & Feedback SystemsC&C
Super Size Me: Interface Size, Identity and Embodiment in Digital Musical Instrument DesignDigital interfaces are shrinking, driven by pressures of mass production and consumer culture, and often accompanied by a discourse of control, precision or convenience. Meanwhile, human bodies remain the same size, and the changing size of interfaces has implications for the formation of user identities. Drawing on embodied cognition, effort and entanglement theories of HCI, we explored the impact of interface size on the co-constitution of humans and technology. We designed an oversized digital musical instrument and invited musicians to use the instrument to create original performances. We found that both the performances and the musicians' self-perception were influenced by the large size of the instrument, shining new light on the ways in which designing technology is designing humans and in turn culture.2022LMLia Mice et al.Queen Mary University of LondonFull-Body Interaction & Embodied InputMusic Composition & Sound Design ToolsCHI
Do People Use Games to Compensate for Psychological Needs During Crises? A Mixed-Methods Study of Gaming During COVID-19 LockdownsDo people use games to cope with adverse life events and crises? Research informed by self-determination theory proposes that people might compensate for thwarted basic psychological needs in daily life by seeking out games that satisfy those lacking needs. To test this, we conducted a preregistered mixed-method survey study (n = 285) on people's gaming behaviours and need states during early stages of the COVID-19 pandemic (May 2020). We found qualitative evidence that gaming was an often actively sought out and successful means of replenishing particular needs, but one that could `backfire' for some through an appraisal process discounting gaming as `unreal'. Meanwhile, contrary to our predictions, the quantitative data showed a ``rich get richer, poor get poorer'' pattern: need satisfaction in daily life positively correlated with need satisfaction in games. We derive methodological considerations and propose three potential explanations for this contradictory data pattern to pursue in future research.2022NBNick Ballou et al.Queen Mary University of LondonGame UX & Player BehaviorMental Health Apps & Online Support CommunitiesCHI
Navigating Imaginaries of DNA-Based Digital Data StorageDNA-based digital data storage technology is hailed as a potential solution for the issues around exponential global data production. However, while the technology continues to strive towards its full commercialization, there is a lack of discourse on how it could be applied to facilitate interactions that are meaningful, ethical, and socially sustainable. As an approach to address this gap, we hosted a series of online workshops, soliciting 15 participants to engage in grounded speculations on plausible futures of DNA data storage. Themes drawn from the resulting imaginaries and discussions were situated within a selection of existing HCI literature, to generate an initial set of design opportunities and challenges for DNA data storage. Early analysis suggests that the system could be designed to 1) facilitate meaningful interactions that are intangible and molecular, and 2) foster better human relationship with more-than-human entities. Furthermore, we highlight the imperative for cross-disciplinary collaborations and pedagogy, to ensure fair and high quality access to the technology.2022RKRaphael Kim et al.Queen Mary University of LondonContext-Aware ComputingTechnology Ethics & Critical HCIDeveloping Countries & HCI for Development (HCI4D)CHI
Moving with the Times: Investigating the Alt-Right Network Gab with Temporal Interaction GraphsGab is an online social network often associated with the alt-right political movement and users barred from other networks. It presents an interesting opportunity for research because near-complete data is available from day one of the network’s creation. In this paper, we investigate the evolution of the user interaction graph, that is the graph where a link represents a user interacting with another user at a given time. We view this graph both at different times and at different timescales. The latter is achieved by using sliding windows on the graph which gives a novel perspective on social network data. The Gab network is relatively slowly growing over the period of months but subject to large bursts of arrivals over hours and days. We identify plausible events that are of interest to the Gab community associated with the most obvious such bursts. The network is characterised by interactions between ‘strangers’ rather than by reinforcing links between ‘friends’. Gab usage follows the diurnal cycle of the predominantly US and Europe based users. At off-peak hours the Gab interaction network fragments into sub-networks with absolutely no interaction between them. A small group of users are highly influential across larger timescales, but a substantial number of users gain influence for short periods of time. Temporal analysis at different timescales gives new insights above and beyond what could be found on static graphs.2021NANaomi A. Arnold et al.Antisocial ComputingCSCW
Exploring Terminology for Perception of Motion in Virtual RealityA key aspect of Virtual Reality (VR) applications is the ability to move in the environment, which relies on the illusion of self-motion to create a good user experience. Self-motion has traditionally been studied in psychophysical studies in which a range of wording has been adopted to describe self-motion. However, it is not clear from current research whether the words used in self-motion studies match study participants' own intuitions about the experience of self-motion. We argue that the terminology used in self-motion studies should be drawn from a participant perspective to improve validity. We undertook an online study involving VR self-motion and card-sorting with 50 participants to examine current self-motion terminology. We found that participants were not familiar with the concept of self-motion and that the virtual scene itself might suggest different terminology. We suggest how studies on motion perception in VR should be designed to better reflect participants' vernacular.2021FSFrancesco Soave et al.Human Pose & Activity RecognitionImmersion & Presence ResearchDIS