Small, Medium, Large? A Meta-Study of Effect Sizes at CHI to Aid Interpretation of Effect Sizes and Power CalculationStatistical reporting, especially of effect sizes, is at the root of many methodological issues in quantitative research at CHI. Effect sizes are necessary for assessing practical relevance of results, a-priori power analysis, and meta-analyses, but currently, they are often not reported. Interpretations in the context of the study and the research field are also rare. To aid to researchers in reporting and contextualizing their effect sizes within their research field as well as choosing effect sizes for power analysis, we conducted a meta-study of quantitative CHI papers. We extracted statistics from all quantitative CHI papers published between 2019-2023 (N=1692). Based on effect sizes and the papers' CCS categories, we present effect size distributions in 12 CHI research fields. Through an additional qualitative analysis of 67 quantitative CHI'23 publications, we identify five categories of approaches that researchers take when interpreting effect size: Comparing test-specific values, assigning size labels, using a statistical or methodological reference frame, comparing different observations and interpreting for the big picture.2025AOAnna-Marie Ortloff et al.University of BonnUser Research Methods (Interviews, Surveys, Observation)Computational Methods in HCIResearch Ethics & Open ScienceCHI
Who is Trusted for a Second Opinion? Comparing Collective Advice from a Medical AI and Physicians in Biopsy Decisions After Mammography ScreeningArtificial Intelligence (AI) is increasingly integrated into clinical practice, but its influence on patient decision-making, particularly when AI and physicians disagree, remains unclear. To examine collective advice, we investigated a breast cancer screening scenario using (1) a qualitative interview study (N=9) and (2) a quantitative experiment (N=339) where participants received either consistent or conflicting biopsy recommendations. Qualitative findings include the need for empathetic care, the importance of patient autonomy, and a desire for a four-eyes principle. Quantitative findings accordingly show that patients generally trust physicians more than AI but still tend to follow AI recommendations due to risk aversion. When both advised a biopsy, 99% adhered; if both advised against it, 25% still proceeded. In conflicting scenarios, 97% followed the physician’s advice, whereas 66% followed the AI if it recommended the biopsy. These results underscore the need for careful interaction design of collective healthcare advice to prevent unnecessary healthcare procedures.2025HDHenrik Detjen et al.Fraunhofer Institute for Digital Medicine MEVISAI-Assisted Decision-Making & AutomationAI Ethics, Fairness & AccountabilityPrivacy by Design & User ControlCHI
"They are responsible for ensuring that I can continue to use the service." Investigating Users' Expectations Towards 2FA Recovery in GermanyTwo-factor authentication is often recommended for increasing online security, and users often follow this by using their phones. If physical items become unavailable, there is a risk of losing access to the account due to missing authentication requirements. In such cases, users need a backup or help from the service. Previous work found no standardized approach to how services address this issue, assist users, or offer backup options. Until now, it is unclear how users handle backups and account recovery and what their expectations towards service providers are. To shed light on this, we conducted 16 interviews and a survey with 95 participants. We found that most had never considered how to access their accounts if the second factor was lost, and only a few had a backup plan. Instead, users often rely on website support, assuming that personal data will help them regain access. We give recommendations for services.2025ETEva Tiefenau et al.Fraunhofer FKIEPrivacy by Design & User ControlPasswords & AuthenticationCHI
Voice Assistants' Accountability through Explanatory Dialogues As voice assistants (VAs) become more advanced leveraging Large Language Models (LLMs) and natural language processing, their potential for accountable behavior expands. Yet, the long-term situational effectiveness of VAs’ accounts when errors occur remains unclear. In our 19-month exploratory study with 19 households, we investigated the impact of an Alexa feature that allows users to inquire about the reasons behind its actions. Our findings indicate that Alexa’s accounts are often single, decontextualized responses that led to users’ alternative repair strategies over the long term, such as turning off the device, rather than initiating a dialogue about what went wrong. Through role-playing workshops, we demonstrate that VA interactions should facilitate explanatory dialogues as dynamic exchanges that consider a range of speech acts, recognizing users’ emotional states and the context of interaction. We conclude by discussing the implications of our findings for the design of accountable VAs.2024FAFatemeh Alizadeh et al.Intelligent Voice Assistants (Alexa, Siri, etc.)Multilingual & Cross-Cultural Voice InteractionExplainable AI (XAI)CUI
Understanding How Parents Deal With the Health Advice They Receive: A Qualitative Study and Implications for the Design of Message-based Health Dissemination Systems for Child HealthMessage-based health information dissemination systems can potentially improve maternal and child health (MCH). By conveying health information to parents, SMS- and chatbot-based systems can support parents’ learning and empower them to make better health decisions for their children. However, there is limited design advice for creating message-based dissemination systems for MCH. To help address this gap, we conducted 14 participatory workshops with 42 parents from Portugal and South Africa, exploring how parents learned to care for their children’s health. Our findings showed how parents reflected on the health advice they received, by assessing the fit of the advice to their child’s characteristics, their values and beliefs, the advice’s feasibility, or the intention and competence of the advice giver. Based on these insights, we propose four design implications for creating message-based health information dissemination systems tailored to parents and their children.2024BFBeatriz Félix et al.Augmentative & Alternative Communication (AAC)Special Education TechnologyMental Health Apps & Online Support CommunitiesDIS
Understanding Feedback in Rhythmic Gymnastics Training: An Ethnographic-Informed Study of a Competition ClassRhythmic Gymnastics is an Olympic sport that demands an exceptional level of expertise. From early age, athletes relentlessly practise exercises until they can flawlessly perform them before an audience and a panel of judges. Technology can potentially support rhythmic gymnasts' training by monitoring gymnasts' exercises and providing feedback on their execution. However, the limited understanding of the training nuances in Rhythmic Gymnastics restricts the development of technologies to support training. Drawing on the observation of training sessions and on interviews with athletes and coaches, this paper uncovers how coaches personalise feedback timing, type, form, format, and quantity, to adapt it to the gymnasts' skill level and type of exercise. Taking stock of our findings, we draw out five implications that can inform the design of systems to support feedback in Rhythmic Gymnastics training.2024LFLeonor Portugal da Fonseca et al.University of CoimbraDance & Body Movement ComputingCHI
Designing for Human Operations on the Moon: Challenges and Opportunities of Navigational HUD Interfaces Future crewed missions to the Moon will face significant environmental and operational challenges, posing risks to the safety and performance of astronauts navigating its inhospitable surface. Whilst head-up displays (HUDs) have proven effective in providing intuitive navigational support on Earth, the design of novel human-spaceflight solutions typically relies on costly and time-consuming analogue deployments, leaving the potential use of lunar HUDs largely under-explored. This paper explores an alternative approach by simulating navigational HUD concepts in a high-fidelity Virtual Reality (VR) representation of the lunar environment. In evaluating these concepts with astronauts and other aerospace experts (n=25), our mixed methods study demonstrates the efficacy of simulated analogues in facilitating rapid design assessments of early-stage HUD solutions. We illustrate this by elaborating key design challenges and guidelines for future lunar HUDs. In reflecting on the limitations of our approach, we propose directions for future design exploration of human-machine interfaces for the Moon.2024LBLeonie Bensch et al.German Aerospace Center (DLR), European Space Agency (ESA)Head-Up Display (HUD) & Advanced Driver Assistance Systems (ADAS)AR Navigation & Context AwarenessCHI
Grand challenges in WaterHCIRecent combinations of interactive technology, humans, and water have resulted in “WaterHCI”. WaterHCI design seeks to complement the many benefits of engagement with the aquatic domain, by offering, for example, augmented reality systems for snorkelers, virtual reality in floatation tanks, underwater musical instruments for artists, robotic systems for divers, and wearables for swimmers. We conducted a workshop in which WaterHCI experts articulated the field’s grand challenges, aiming to contribute towards a systematic WaterHCI research agenda and ultimately advance the field.2024FMFlorian Floyd Mueller et al.Monash UniversityDigital Art Installations & Interactive PerformanceDance & Body Movement ComputingCHI
Becoming an Online Wine Taster: An Ethnographic Study on the Digital Mediation of TasteThere has been a growing interest in taste research in the HCI and CSCW communities. However, the focus is more on stimulating the senses, while the socio-cultural aspects have received less attention. However, individual taste perception is mediated through social interaction and collective negotiation and is not only dependent on physical stimulation. Therefore, we study the digital mediation of taste by drawing on ethnographic research of four online wine tastings and one self-organized event. Hence, we investigated the materials, associated meanings, competences, procedures, and engagements that shaped the performative character of tasting practices. We illustrate how the tastings are built around the taste-making process and how online contexts differ in providing a more diverse and distributed environment. We then explore the implications of our findings for the further mediation of taste as a social and democratized phenomenon through online interaction.2023JBJenny Berkholz et al.Online ExperiencesCSCW
Less About Privacy: Revisiting a Survey about the German COVID-19 Contact Tracing AppThe release of COVID-19 contact tracing apps was accompanied by a heated public debate with much focus on privacy concerns, e.g., possible government surveillance. Many papers studied people's intended behavior to research potential features and uptake of the apps. Studies in Germany conducted before the app's release, such as that by Häring et al., showed that privacy was an important factor in the intention to install the app. We conducted a follow-up study two months post-release to investigate the intention-behavior-gap, see how attitudes changed after the release, and capture reported behavior. Analyzing a quota sample (n=837) for Germany, we found that fewer participants mentioned privacy concerns post-release, whereas utility now plays a greater role. We provide further evidence that the results of intention-based studies should be handled with care when used for prediction purposes.2023MHMaximilian Häring et al.University of BonnPrivacy by Design & User ControlPrivacy Perception & Decision-MakingContent Moderation & Platform GovernanceCHI
A Usability Evaluation of AFL and libFuzzer with CS StudentsIn top-tier companies and academia, fuzzing has established itself as a valuable tool for finding bugs. It is a tool created by experts for experts, and a lot of research is being invested into improving the power of fuzzing. However, the usability of fuzzing has not received much attention yet. To alleviate this, we evaluated the usability of two popular fuzzers: AFL and libFuzzer. In our fuzzing study, 47 computer science students each worked up to 20 hours in total. We found significant usability challenges for both fuzzers leading to only 17 participants who were able to finish all tasks. Even the successful participants struggled with some of the necessary steps and found them complex and confusing. While on the whole, AFL fared better than libFuzzer, both fuzzers have strengths and weaknesses and can be improved based on our results.2023SPStephan Plöger et al.Fraunhofer FKIEMental Health Apps & Online Support CommunitiesComputational Methods in HCICHI
Elements of XR Prototyping: Characterizing the Role and Use of Prototypes in Augmented and Virtual Reality DesignCurrent research in augmented, virtual, and mixed reality (XR) reveals a lack of tool support for designing and, in particular, prototyping XR applications. While recent tools research is often motivated by studying the requirements of non-technical designers and end-user developers, the perspective of industry practitioners is less well understood. In an interview study with 17 practitioners from different industry sectors working on professional XR projects, we establish the design practices in industry, from early project stages to the final product. To better understand XR design challenges, we characterize the different methods and tools used for prototyping and describe the role and use of key prototypes in the different projects. We extract common elements of XR prototyping, elaborating on the tools and materials used for prototyping and establishing different views on the notion of fidelity. Finally, we highlight key issues for future XR tools research.2022VKVeronika Krauß et al.University of Siegen, Bonn-Rhein Sieg University of Applied ScienceMixed Reality WorkspacesPrototyping & User TestingCHI
Alexa, We Need to Talk: A Data Literacy Approach on Voice AssistantsVoice assistants (VA) collect data about users’ daily life including interactions with other connected devices, musical preferences, and unintended interactions. While users appreciate the convenience of VAs, their understanding and expectations of data collection by vendors are often vague and incomplete. By making the collected data explorable for consumers, our research-through-design approach seeks to unveil design resources for fostering data literacy and help users in making better informed decisions regarding their use of VAs. In this paper, we present the design of an interactive prototype that visualizes the conversations with VAs on a timeline and provides end users with basic means to engage with data, for instance allowing for filtering and categorization. Based on an evaluation with eleven households, our paper provides insights on how users reflect upon their data trails and presents design guidelines for supporting data literacy of consumers in the context of VAs.2021DPDominik Pins et al.Intelligent Voice Assistants (Alexa, Siri, etc.)Privacy Perception & Decision-MakingDIS
Avoiding Reactions Outside the Home: Challenges, Strategies, and Opportunities to Enhance Dining Out Experiences of People with Food HypersensitivitiesPeople with food hypersensitivities experience adverse reactions when eating certain foods and thus need to adapt their diet. When dining out, the challenge is greater as people entrust the care of their allergy, intolerance, or celiac disease, in the hands of staff who might not have enough knowledge to appropriately care for them. This interview study explored how people with food hypersensitivities avoid reactions while eating out, to inspire future digital technology design. Our findings show the social and emotional impact of food hypersensitivities and how people practically cope by investigating restaurants' safety precautions, correcting orders, or even educating restaurants' staff. We discuss our findings against the experiences of other people living with chronic conditions and offer design opportunities for digital technologies to enhance dining out experiences of people with food hypersensitivities.2021FNFrancisco Nunes et al.Fraunhofer Portugal AICOSMental Health Apps & Online Support CommunitiesDiet Tracking & Nutrition ManagementCHI
Current Practices, Challenges, and Design Implications for Collaborative AR/VR Application DevelopmentAugmented/Virtual Reality (AR/VR) is still a fragmented space to design for due to the rapidly evolving hardware, the interdisciplinarity of teams, and a lack of standards and best practices. We interviewed 26 professional AR/VR designers and developers to shed light on their tasks, approaches, tools, and challenges. Based on their work and the artifacts they generated, we found that AR/VR application creators fulfill four roles: concept developers, interaction designers, content authors, and technical developers. One person often incorporates multiple roles and faces a variety of challenges during the design process from the initial contextual analysis to the deployment. From analysis of their tool sets, methods, and artifacts, we describe critical key challenges. Finally, we discuss the importance of prototyping for the communication in AR/VR development teams and highlight design implications for future tools to create a more usable AR/VR tool chain.2021VKVeronika Krauß et al.Fraunhofer Institute for Applied Information Technology FIT, University of SiegenSocial & Collaborative VRMixed Reality WorkspacesCHI
ShiSha: Enabling Shared Perspective With Face-to-Face Collaboration Using Redirected Avatars in Virtual RealityThe importance of remote collaboration grows in an interconnected world as the reasons to avoid travel increase. The spatial rendering and collaboration capabilities of virtual and augmented reality systems are well suited for tasks such as support or training. Users can take a shared perspective to build a common understanding. Also, users may engage in face-to-face cooperation to support interpersonal communication. However, a shared perspective and face-to-face collaboration are both desirable but naturally exclude each other. We place all users at the same location to provide a shared perspective. To avoid overlapping body parts, the avatars of the other connected users are shifted to the side. A redirected body pose modification corrects the resulting inconsistencies. The implemented system is compared to a baseline of two users standing in the same location and working with overlapping avatars. The results of a user study show that the proposed modifications provide an easy to use, efficient collaboration and yield higher co-presence and the feeling of teamwork. Applying redirection techniques to other users opens up novel ways to increase social presence for local or remote collaboration.2020AHAdrian H. Hoppe et al.Videos, Live Streaming, and VRCSCW
On Conducting Security Developer Studies with CS Students: Examining a Password-Storage Study with CS Students, Freelancers, and Company DevelopersEcological validity is a major concern in usable security studies with developers. Many studies are conducted with computer science (CS) students out of convenience, since recruiting professional software developers in sufficient numbers is very challenging. In a password-storage study, Naiakshina et al. (CHI'19) showed that CS students behave similarly to freelance developers recruited online. While this is a promising result for conducting developer studies with students, an open question remains: Do professional developers employed in companies behave similarly as well? To provide more insight into the ecological validity of recruiting students for security developer studies, we replicated the study of Naiakshina et al. with developers from diverse companies in Germany. We found that developers employed in companies performed better than students and freelancers in a direct comparison. However, treatment effects were found to be significant in all groups; the treatment effects on CS students also held for company developers.2020ANAlena Naiakshina et al.University of BonnPasswords & AuthenticationUser Research Methods (Interviews, Surveys, Observation)CHI
How to Work in the Car of the Future? A Neuroergonomical Study Assessing Concentration, Performance and Workload Based on Subjective, Behavioral and Neurophysiological InsightsAutonomous driving provides new opportunities for the use of time during a car ride. One such important scenario is working. We conducted a neuroergonomical study to compare three configurations of a car interior (based on lighting, visual stimulation, sound) regarding their potential to support productive work. We assessed participants' concentration, performance and workload with subjective, behavioral and EEG measures while they carried out two different concentration tasks during simulated autonomous driving. Our results show that a configuration with a large-area, bright light with high blue components, and reduced visual and auditory stimuli promote performance, quality, efficiency, increased concentration and lower cognitive workload. Increased visual and auditory stimulation paired with linear, darker light with very few blue components resulted in lower performance, reduced subjective concentration, and higher cognitive workload, but did not differ from a normal car configuration. Our multi-method approach thus reveals possible car interior configurations for an ideal workspace.2019KPKathrin Pollmann et al.University of Stuttgart & Fraunhofer Institute for Industrial Engineering IAOHead-Up Display (HUD) & Advanced Driver Assistance Systems (ADAS)Knowledge Worker Tools & WorkflowsCHI
Security Managers Are Not The Enemy EitherSecurity managers are leading employees whose decisions shape security measures and thus influence the everyday work of all users in their organizations. To understand how security managers handle user requirements and behavior, we conducted semi-structured interviews with seven security managers from large-scale German companies. Our results indicate that due to the absence of organizational structures that include users into security development processes, security managers unintentionally obtain a negative view on users. Their distrust towards users leads to the creation of technical security measures that cannot be influenced by users in any way. However, as previous research has repeatedly shown, rigid security measures lead to frustration and discouragement of users, and also to creative (but usually insecure) methods of security circumvention. We conclude that in order to break through this vicious cycle, security managers need organizational structures, methods and tools that facilitate systematic feedback from users.2019LRLena Reinfelder et al.Friedrich-Alexander-Universität Erlangen-NürnbergPrivacy by Design & User ControlCybersecurity Training & AwarenessCHI
What Moves Players? Visual Data Exploration of Twitter and Gameplay DataIn recent years, microblogging platforms have not only become an important communication channel for the game industry to generate and uphold audience interest but also a rich resource for gauging player opinion. In this paper we use data gathered from Twitter to examine which topics matter to players and to identify influential members of a game's community. By triangulating in-game data with Twitter activity we explore how tweets can provide contextual information for understanding fluctuations in in-game activity. To facilitate analysis of the data we introduce a visual data exploration tool and use it to analyze tweets related to the game Destiny. In total, we collected over one million tweets from about 250,000 users over a 14-month period and gameplay data from roughly 3,500 players over a six-month period.2018CDChristian Drescher et al.Vienna University of TechnologyInteractive Data VisualizationMultiplayer & Social GamesCHI