Seeing the Sound: Supporting Musical Collaboration with Augmented RealityIn musical collaboration, digital musical instruments often hinder effective communication and engagement by restricting visibility and limiting gestural and non-verbal interactions. These challenges reduce musicians’ situational awareness and complicate cohesive performance. To address this, we developed a head-mounted augmented reality (AR) system to enhance collaborative musical experiences by visualising musicians’ hand movements, eye gaze positions, and instrument interactions in real-time. We conducted a user study involving four pairs of musicians performing live music using different AR interface configurations. The results suggest that the AR system can enhance situational awareness and assist collaboration, as reflected in questionnaire responses. Interviews indicated that real-time visualisations of bodily movements and interactions helped participants better understand the collaborative process and anticipate their collaborators’ actions. These findings point to the potential of AR-assisted visualisation to support creative collaboration by tailoring visual information to different needs. Future research could explore its application in broader contexts of real-time creative cooperation.2025YWYichen Wang et al.Social & Collaborative VRAR Navigation & Context AwarenessImmersion & Presence ResearchC&C
Designing with decolonial intent: Towards a decolonial archive in resistance to epistemicideThis paper follows a trans-disciplinary and trans-cultural arts research endeavour which seeks to utilise the restitution of neglected archival materials to engage the social and cultural trajectory of the villages and nation from which that material and intangible heritage was taken, stolen, destroyed, lost, or diminished. The paper engages with tensions in colonial and decolonial design of digital heritage between the potential for counter-histories and imaginaries on the one-hand and the colonial impulse of computing and its logics on the other. Through the research through design activities formed with a decolonial praxiology, we explore how the systems, practices and technologies of archival practices in this project develop an ethics of knowledge-making that neither satisfies or diminishes decolonial intent. We tentatively argue for approaches to decolonial design that are accounted for in local and pragmatic modes of knowledge making that are delinked from globalised and abstracted systems that otherwise repress them.2025RCRob Comber et al.Technology Ethics & Critical HCIDesign FictionMuseum & Cultural Heritage DigitizationDIS
LumaDreams: Designing Positive Dream Meaning-Making for Daily EmpowermentDreams contribute to cognitive and emotional health, yet tools for everyday dream engagement remain largely underexplored outside clinical settings. In this paper, we introduce LumaDreams, a mobile application designed to foster daily empowerment through positive dream transformation using generative AI. Informed by meaning-making theories, LumaDreams enables users to journal dreams through sketches and text, which are then transformed into positive images and stories for users to revisit and reflect on. We conducted a mixed-method study with 14 participants over 14 days. Our findings show that LumaDreams strengthened participants’ daily empowerment through cognitive and emotional shifts that arise from the positive meaning-making process. Qualitative insights further revealed how users’ perceptions and trust of AI-driven dream transformation were shaped through their interactions. In conclusion, we propose an inspiring approach that enables users to co-create positive meanings in dream experiences with generative AI, promoting cognitive and emotional shifts, fostering positive mindsets, and ultimately strengthening daily empowerment.2025BLBolin Lyu et al.Southeast University, School of Computer Science and EngineeringGenerative AI (Text, Image, Music, Video)Mental Health Apps & Online Support CommunitiesCHI
Snap, Sweat, and Sketch: Designing Home Exercise Experiences for Augmented Reality Head-mounted DisplaysAugmented Reality (AR) head-mounted displays (HMDs) offer potential for more inclusive and immersive exercising and exergaming experiences at home. Previous work found that augmenting home objects can create more engaging exercise experiences and identified various home objects that can be augmented to facilitate different exercises. However, it is unclear how these objects can be augmented to enhance exercising and tailored based on the exercise. We conducted a multi-part study involving a design activity using Snapchat and focus group discussion with 28 participants. We present five themes relating to participants' preferences for the augmentation of home objects for exercising, and identify and discuss key guidelines that designers and researchers should consider when augmenting home objects. Our results provide designers with guidelines and ideas for the augmentation of four different exercises, and advance the foundation for future work developing home-based exergaming through AR HMDs to increase people's physical activity levels.2025MAMichelle Adiwangsa et al.Australian National University, School of ComputingAR Navigation & Context AwarenessFitness Tracking & Physical Activity MonitoringInteractive Narrative & Immersive StorytellingCHI
Grand challenges in CyclingHCICycling Human-Computer Interaction (CyclingHCI) refers to the study and design of user interfaces and interactions between bicycles and riders in the context of cycling-related experiences. To date, however, there has yet to be a structured agenda for CyclingHCI to clarify the immediate challenges researchers should address next and facilitate the advancement of the field. To advance the development of CyclingHCI, we employed expert sessions with three CyclingHCI experts responsible for the design, development, evaluation, and reflection on the societal implications of 18 CyclingHCI systems. Our analysis led us to 10 grand challenges with design opportunities and considerations grouped as (1) Pushing the technological boundaries for cycling, (2) Understanding and protecting cyclists, and (3) Spatially situated cycling interaction. Our findings provide practical implications for research and practice in designing for CyclingHCI, with which we aim to advance CyclingHCI and enrich the cycling experience through the safe integration of technology.2024AMAndrii Matviienko et al.Micromobility (E-bike, E-scooter) InteractionDIS
From Fitting Participation to Forging Relationships: The Art of Participatory MLParticipatory machine learning (ML) encourages the inclusion of end users and people affected by ML systems in design and development processes. We interviewed 18 participation brokers—individuals who facilitate such inclusion and transform the products of participants' labour into inputs for an ML artefact or system—across a range of organisational settings and project locations. Our findings demonstrate the inherent challenges of integrating messy contextual information generated through participation with the structured data formats required by ML workflows and the uneven power dynamics in project contexts. We advocate for evolution in the role of brokers to more equitably balance value generated in Participatory ML projects for design and development teams with value created for participants. To move beyond 'fitting' participation to existing processes and empower participants to envision alternative futures through ML, brokers must become educators and advocates for end users, while attending to frustration and dissent from indirect stakeholders.2024NCNed Cooper et al.Australian National UniversityHuman-LLM CollaborationParticipatory DesignCHI
Exploring Opportunities for Augmenting Homes to Support ExercisingAlthough exercising at home has benefits, it is not always engaging or motivating. Augmented Reality (AR) head-mounted displays (HMDs) offer the potential to make in-home exercising and exergaming more inclusive and immersive, but there is limited research investigating how such systems can be designed. We employed a participatory design approach involving semi-structured interviews to investigate how homes can be augmented to facilitate exercising experiences. We developed 10 recommendations for developing home-based exercising experiences using AR HMDs. Our results further contribute to the existing body of research on the use of AR for exercising, home applications, and everyday objects by presenting the first foundational study investigating the wide range of exercises that can be supported through AR HMDs in home environments and the different ways home elements may support these exercises, and laying the groundwork for future work developing home-based exergaming through AR HMDs to increase people's physical activity levels.2024MAMichelle Adiwangsa et al.Australian National UniversityAR Navigation & Context AwarenessFitness Tracking & Physical Activity MonitoringCHI
A Scoping Study of Evaluation Practices for Responsible AI Tools: Steps Towards Effectiveness EvaluationsResponsible design of AI systems is a shared goal across HCI and AI communities. Responsible AI (RAI) tools have been developed to support practitioners to identify, assess, and mitigate ethical issues during AI development. These tools take many forms (e.g., design playbooks, software toolkits, documentation protocols). However, research suggests that use of RAI tools is shaped by organizational contexts, raising questions about how effective such tools are in practice. To better understand how RAI tools are—and might be—evaluated, we conducted a qualitative analysis of 37 publications that discuss evaluations of RAI tools. We find that most evaluations focus on usability, while questions of tools’ effectiveness in changing AI development are sidelined. While usability evaluations are an important approach to evaluate RAI tools, we draw on evaluation approaches from other fields to highlight developer- and community-level steps to support evaluations of RAI tools’ effectiveness in shaping AI development practices and outcomes.2024GBGlen Berman et al.Australian National UniversityAI-Assisted Decision-Making & AutomationAI Ethics, Fairness & AccountabilityAlgorithmic Fairness & BiasCHI
Reading Between the Lines: Identifying the Linguistic Markers of Anhedonia for the Stratification of DepressionStratifying depressed individuals may help to improve recovery rates by identifying the subgroups who would benefit from targeted treatments. Detecting depressed individuals with prominent anhedonia (i.e. lack of pleasure) may be one effective approach, given these individuals experience poorer treatment outcomes. This paper explores the linguistic features associated with anhedonia among depressed adults. Over 9 weeks, 218 individuals with depressive symptoms completed a fortnightly psychometric measure of depression (PHQ-9) and provided text data (SMS, social media posts, expressive essays, emotion diaries, personal letters). Linguistic features were examined using LIWC-22. Greater use of discrepancy words was significantly associated with higher anhedonia, but in SMS data only. Machine learning showed some utility for predicting increased anhedonia, with discrepancy words the most important linguistic feature in the model. Discrepancy words were not found to be associated with overall depression scores. These results suggest that this linguistic feature may show some promise for the stratification of anhedonic depression.2024BOBridianne O'Dea et al.University of New South WalesCognitive Impairment & Neurodiversity (Autism, ADHD, Dyslexia)CHI
Multi-Embodiment and Robot Identity: A Scoping ReviewMulti-embodied agents can have both physical and virtual bodies, moving between real and virtual environments to meet user needs, embodying robots or virtual agents alike to support extended human-agent relationships. As a design paradigm, multi-embodiment offers potential benefits to improve communication and access to artificial agents, but there are still many unknowns in how to design these kinds of systems. This paper presents the results of a scoping review of the multi-embodiment and robot identity research, aimed at consolidating the existing evidence and identifying knowledge gaps. Based on our review, we identify key research themes of: multi-embodied systems, identity design, human-agent interaction, environment and context, trust, and information and control. We also identify 16 key research challenges and 12 opportunities for future research.2024KKKarla Bransky Kelly et al.Identity & Avatars in XRSocial Robot InteractionHuman-Robot Collaboration (HRC)HRI
Unveiling the Tricks: Automated Detection of Dark Patterns in Mobile ApplicationsMobile apps bring us many conveniences, such as online shopping and communication, but some use malicious designs called dark patterns to trick users into doing things that are not in their best interest. Many works have been done to summarize the taxonomy of these patterns and some have tried to mitigate the problems through various techniques. However, these techniques are either time-consuming, not generalisable or limited to specific patterns. To address these issues, we propose UIGuard, a knowledge-driven system that utilizes computer vision and natural language pattern matching to automatically detect a wide range of dark patterns in mobile UIs. Our system relieves the need for manually creating rules for each new UI/app and covers more types with superior performance. In detail, we integrated existing taxonomies into a consistent one, conducted a characteristic analysis and distilled knowledge from real-world examples and the taxonomy. Our UIGuard consists of two components, Property Extraction and Knowledge-Driven Dark Pattern Checker. We collected the first dark pattern dataset, which contains 4,999 benign UIs and 1,353 malicious UIs of 1,660 instances spanning 1,023 mobile apps. Our system achieves a superior performance in detecting dark patterns (micro averages: 0.82 in precision, 0.77 in recall, 0.79 in F1 score). A user study involving 58 participants further showed that UIGuard significantly increases users' knowledge of dark patterns. We demonstrated potential use cases of our work, which can benefit different stakeholders, and serve as a training tool for raising awareness of dark patterns2023JCJieshan Chen et al.Dark Patterns RecognitionUIST
The Shapes of the Fourth Estate During the Pandemic: Profiling COVID-19 News Consumption in Eight CountriesNews media is often referred to as the Fourth Estate, a recognition of its political power. New understandings of how media shape political beliefs and influence collective behaviors are urgently needed in an era when public opinion polls do not necessarily reflect election results and users influence each other in real-time under algorithm-mediated content personalization. In this work, we measure not only the average but also the distribution of audience political leanings for different media across different countries. The methodological components of these new measurements include a high-fidelity COVID-19 tweet dataset; high-precision user geolocation extraction; and user political leaning estimated from the within-country retweet networks involving local politicians. We focus on geolocated users from eight countries, profile user leaning distribution for each country, and analyze bridging users who have interactions across multiple countries. Except for France and Turkey, we observe consistent bi-modal user leaning distributions in the other six countries, and find that cross-country retweeting behaviors do not oscillate across the partisan divide. More importantly, this study contributes a new set of media bias estimates by averaging the leaning scores of users who share the URLs from media domains. Through two validations, we find that the new average audience leaning scores strongly correlate with existing media bias scores. Lastly, we profile the COVID-19 news consumption by examining the audience leaning distribution for top media in each country, and for selected media across all countries. Those analyses help answer questions such as: Does center media Reuters have a more balanced audience base than partisan media CNN and Fox News in the US? Does far-right media Breitbart attract any left-leaning readers in any countries? Does CNN reach a more balanced audience base in the US than in UK and Spain? In sum, our data-driven methods allow us to study media that are not often collected in editor-curated media bias reporting, especially in non-English-speaking countries. We hope that such cross-country research would inform media outlets of their effectiveness and audience bases in different countries, inform non-government and research organizations about the country-specific media audience profiles, and inform individuals to reflect on our day-to-day media diet.2023CYCai Yang et al.COVID-19 + CSCWCSCW
Embodying an Interactive AI for Dance Through Movement IdeationWhat expectations exist in the minds of dancers when interacting with a generative machine learning model? During two workshop events, experienced dancers explore these expectations through improvisation and role-play, embodying an imagined AI-dancer. Through discussions with the participants we identify a variety of ways an AI-dancer might be useful to human dancers. The dancers explored how intuited flow, shared images, and the concept of a human replica might work in their imagined AI-human interaction. Our findings challenge existing assumptions about what is desired from generative models of dance, such as expectations of realism, and how such systems should be evaluated. We further advocate that such models should celebrate non-human artefacts, focus on the potential for serendipitous moments of discovery, and that dance practitioners should be included in their development. Our concrete suggestions show how our findings can be adapted into the development of improved generative and interactive machine learning models for dancers' creative practice.2023BWBenedikte Wallace et al.Generative AI (Text, Image, Music, Video)Digital Art Installations & Interactive PerformanceDance & Body Movement ComputingC&C
Learning Embodied Sound-Motion Mappings: Evaluating AI-Generated Dance ImprovisationThrough dance, a wide range of emotions can be expressed. As virtual agents and robots continue to become part of our daily lives, the need for them to efficiently convey emotion and intent increases. When trained to dance, to what extent can AI learn to model the tacit mappings between sound and motion? Here, we explore the creative capacity of a generative model trained on 3D motion capture recordings of improvised dance. We perform a perceptual judgment experiment wherein respondents rate movement generated by our model as well as human performances. While the sound-motion mappings remain somewhat elusive, particularly when compared to examples of human dance, our study shows that in certain aspects related to perceived dance-likeness and expressivity, the model successfully mimics human dance. By employing a perceptual study to evaluate our generative model, we aim to further our ability to understand the affordances and limitations of creative AI.2022BWBenedikte Wallace et al.Generative AI (Text, Image, Music, Video)Dance & Body Movement ComputingC&C
A Systematic Review and Thematic Analysis of Community-Collaborative Approaches to Computing ResearchHCI researchers have been gradually shifting attention from individual users to communities when engaging in research, design, and system development. However, our field has yet to establish a cohesive, systematic understanding of the challenges, benefits, and commitments of community-collaborative approaches to research. We conducted a systematic review and thematic analysis of 47 computing research papers discussing participatory research with communities for the development of technological artifacts and systems, published over the last two decades. From this review, we identified seven themes associated with the evolution of a project: from establishing community partnerships to sustaining results. Our findings suggest that several tensions characterize these projects, many of which relate to the power and position of researchers, and the computing research environment, relative to community partners. We discuss the implications of our findings and offer methodological proposals to guide HCI, and computing research more broadly, towards practices that center communities.2022NCNed Cooper et al.Australian National UniversityCommunity Engagement & Civic TechnologyParticipatory DesignResearch Ethics & Open ScienceCHI
Logic Bonbon: Exploring Food as Computational ArtifactIn recognition of food’s significant experiential pleasures, culinary practitioners and designers are increasingly exploring novel combinations of computing technologies and food. However, despite much creative endeavors, proposals and prototypes have so far largely maintained a traditional divide, treating food and technology as separate entities. In contrast, we present a “Research through Design” exploration of the notion of food as computational artifact: wherein food itself is the material of computation. We describe the Logic Bonbon, a dessert that can hydrodynamically regulate its flavor via a fluidic logic system. Through a study of experiencing the Logic Bonbon and reflection on our design practice, we offer a provisional account of how food as computational artifact can mediate new interactions through a novel approach to food-computation integration, that promotes an enriched future of Human-Food Interaction.2022JDJialin Deng et al.Monash UniversityUbiquitous ComputingFood Culture & Food InteractionCHI
Towards Complete Icon Labeling in Mobile ApplicationsAccurately recognizing icon types in mobile applications is integral to many tasks, including accessibility improvement, UI design search, and conversational agents. Existing research focuses on recognizing the most frequent icon types, but these technologies fail when encountering an unrecognized low-frequency icon. In this paper, we work towards complete coverage of icons in the wild. After annotating a large-scale icon dataset (327,879 icons) from iPhone apps, we found a highly uneven distribution: 98 common icon types covered 92.8% of icons, while 7.2% of icons were covered by more than 331 long-tail icon types. In order to label icons with widely varying occurrences in apps, our system uses an image classification model to recognize common icon types with an average of 3,000 examples each (96.3% accuracy) and applies a few-shot learning model to classify long-tail icon types with an average of 67 examples each (78.6% accuracy). Our system also detects contextual information that helps characterize icon semantics, including nearby text (95.3% accuracy) and modifier symbols added to the icon (87.4% accuracy). In a validation study with workers (n=23), we verified the usefulness of our generated icon labels. The icon types supported by our work cover 99.5% of collected icons, improving on the previously highest 78% coverage in icon classification work.2022JCJieshan Chen et al.Australian National UniversityHuman-LLM CollaborationRecommender System UXCHI
Learning Embodied Sound-Motion Mappings: Evaluating AI-Generated Dance ImprovisationThrough dance, a wide range of emotions can be expressed. As virtual agents and robots continue to become part of our daily lives, the need for them to efficiently convey emotion and intent increases. When trained to dance, to what extent can AI learn to model the tacit mappings between sound and motion? Here, we explore the creative capacity of a generative model trained on 3D motion capture recordings of improvised dance. We perform a perceptual judgment experiment wherein respondents rate movement generated by our model as well as human performances. While the sound-motion mappings remain somewhat elusive, particularly when compared to examples of human dance, our study shows that in certain aspects related to perceived dance-likeness and expressivity, the model successfully mimics human dance. By employing a perceptual study to evaluate our generative model, we aim to further our ability to understand the affordances and limitations of creative AI.2021BWBenedikte Wallace et al.Generative AI (Text, Image, Music, Video)Dance & Body Movement ComputingC&C
Diagramming Working Field Theories for Design in the HCI ClassroomHCI has historically provided little support for moving from fieldwork insights or theories to design outcomes. Having witnessed many students struggle and then justify their designs with a form of marketing hype, we developed a supporting approach of “field theories”. A field theory is a working theory about salient interactions in a particular domain and sensitizing concepts in order to frame design investigations. It is presented visually in a field theory diagram to support succinct communication and critique. Studying use of design prototypes that have been informed by a field theory helps to reflect upon and refine the theory. In this paper we present examples from our HCI classes and reflections based on interviews with students. We discuss how field theories offer an orientation in the spirit of a ‘bricoleur’ who harnesses elements of theory and practice to produce deeper understandings and more fitting outcomes for the task at hand.2021BPBernd Ploderer et al.Queensland University of Technology (QUT)User Research Methods (Interviews, Surveys, Observation)CHI
Dark Patterns and the Legal Requirements of Consent Banners: An Interaction Criticism PerspectiveUser engagement with data privacy and security through consent banners has become a ubiquitous part of interacting with internet services. While previous work has addressed consent banners from either interaction design, legal, and ethics-focused perspectives, little research addresses the connections among multiple disciplinary approaches, including tensions and opportunities that transcend disciplinary boundaries. In this paper, we draw together perspectives and commentary from HCI, design, privacy and data protection, and legal research communities, using the language and strategies of "dark patterns" to perform an interaction criticism reading of three different types of consent banners. Our analysis builds upon designer, interface, user, and social context lenses to raise tensions and synergies that arise together in complex, contingent, and conflicting ways in the act of designing consent banners. We conclude with opportunities for transdisciplinary dialogue across legal, ethical, computer science, and interactive systems scholarship to translate matters of ethical concern into public policy.2021CGColin M. Gray et al.Purdue UniversityPrivacy by Design & User ControlDark Patterns RecognitionCHI