MEDebiaser: A Human-AI Feedback System for Mitigating Bias in Multi-label Medical Image ClassificationMedical images often contain multiple labels with imbalanced distributions and co-occurrence, leading to bias in multi-label medical image classification. Close collaboration between medical professionals and machine learning practitioners has significantly advanced medical image analysis. However, traditional collaboration modes struggle to facilitate effective feedback between physicians and AI models, as integrating medical expertise into the training process via engineers can be time-consuming and labor-intensive. To bridge this gap, we introduce MEDebiaser, an interactive system enabling physicians to directly refine AI models using local explanations. By combining prediction with attention loss functions and employing a customized ranking strategy to alleviate scalability, MEDebiaser allows physicians to mitigate biases without technical expertise, reducing reliance on engineers, and thus enhancing more direct human-AI feedback. Our mechanism and user studies demonstrate that it effectively reduces biases, improves usability, and enhances collaboration efficiency, providing a practical solution for integrating medical expertise into AI-driven healthcare.2025SSShaohan Shi et al.Brain-Computer Interface (BCI) & NeurofeedbackExplainable AI (XAI)AI-Assisted Decision-Making & AutomationUIST
PaperBridge: Crafting Research Narratives through Human-AI Co-ExplorationResearchers frequently need to synthesize their own publications into coherent narratives that demonstrate their scholarly contributions. To suit diverse communication contexts, exploring alternative ways to organize one’s work while maintaining coherence is particularly challenging, especially in interdisciplinary fields like HCI where individual researchers' publications may span diverse domains and methodologies. In this paper, we present PaperBridge, a human–AI co-exploration system informed by a formative study and content analysis. PaperBridge assists researchers in exploring diverse perspectives for organizing their publications into coherent narratives. At its core is a bi-directional analysis engine powered by large language models, supporting iterative exploration through both top-down user intent (e.g., determining organization structure) and bottom-up refinement on narrative components (e.g., thematic paper groupings). Our user study (N=12) demonstrated PaperBridge's usability and effectiveness in facilitating the exploration of alternative research narratives. Our findings also provided empirical insights into how interactive systems can scaffold academic communication tasks.2025RZRunhua ZHANG et al.Human-LLM CollaborationData StorytellingComputational Methods in HCIUIST
Understood: Real-Time Communication Support for Adults with ADHD Using Mixed RealityAdults with Attention Deficit Hyperactivity Disorder (ADHD) often experience communication challenges, primarily due to executive dysfunction and emotional dysregulation, even after years of social integration. While existing interventions predominantly target children through structured or intrusive methods, adults lack tools that translate clinical strategies into daily communication support. To address this gap, we present Understood, a Mixed Reality (MR) system implemented on Microsoft HoloLens 2, designed to assist adults with ADHD in real-world communication. Through formative semi-structured interviews and a design workshop, we identified critical communication barriers and derived design goals for the system. Understood combines three key features: (1) real-time conversation summarization to reduce cognitive load, (2) context-aware subsequent word suggestions during moments of disfluency, and (3) topic shifting detection and reminding to mitigate off-topic transitions. A within-subjects user study and expert interviews demonstrate that Understood effectively supports communication with high usability, offering a complement to therapist-mediated interventions.2025SZShizhen Zhang et al.Mixed Reality WorkspacesCognitive Impairment & Neurodiversity (Autism, ADHD, Dyslexia)UIST
StratIncon Detector: Analyzing Strategy Inconsistencies Between Real-Time Strategy and Preferred Professional Strategy in MOBA EsportsMOBA (Multiplayer Online Battle Arena) games require a delicate interplay of strategic planning and real-time decision-making, particularly in professional esports, where players exhibit varying levels of skill and strategic insight. While team strategies have been widely studied, analyzing inconsistencies in professional matches remains a significant challenge. The complexity lies in defining and quantifying the difference between real-time and preferred professional strategies, as well as understanding the disparities between them. Establishing direct causal links between specific strategic decisions and game outcomes also demands a comprehensive analysis of the entire match progression. To tackle these challenges, we present the StratIncon Detector, a visual analytics system designed to assist professional players and coaches in efficiently identifying strategic inconsistencies. The system detects real-time strategies, predicts preferred professional strategies, extracts relevant human factors, and uncovers their impact on subsequent game phases. Findings from a case study, a user study with 24 participants, and expert interviews suggest that, compared to traditional methods, the StratIncon Detector enables users to more comprehensively and efficiently identify inconsistencies, infer their causes, evaluate their effects on subsequent game outcomes, and gain deeper insights into team collaboration—ultimately enhancing future teamwork.2025RMRuofei Ma et al.Game UX & Player BehaviorMultiplayer & Social GamesIUI
TSConnect: An Enhanced MOOC Platform for Bridging Communication Gaps Between Instructors and Students in Light of the Curse of KnowledgeKnowledge dissemination in educational settings is profoundly influenced by the curse of knowledge, a cognitive bias that causes experts to underestimate the challenges faced by learners due to their own in-depth understanding of the subject. This bias can hinder effective knowledge transfer and pedagogical effectiveness, and may be exacerbated by inadequate instructor-student communication. To encourage more effective feedback and promote empathy, we introduce TSConnect, a bias-aware, adaptable interactive MOOC (Massive Open Online Course) learning system, informed by a need-finding survey involving 129 students and 6 instructors. TSConnect integrates instructors, students, and Artificial Intelligence (AI) into a cohesive platform, facilitating diverse and targeted communication channels while addressing previously overlooked information needs. A notable feature is its dynamic knowledge graph, which enhances learning support and fosters a more interconnected educational experience. We conducted a between-subjects user study with 30 students comparing TSConnect to a baseline system. Results indicate that TSConnect significantly encourages students to provide more feedback to instructors. Additionally, interviews with 4 instructors reveal insights into how they interpret and respond to this feedback, potentially leading to improvements in teaching strategies and the development of broader pedagogical skills.2025QLQianyu Liu et al.Online Learning & MOOC PlatformsIntelligent Tutoring Systems & Learning AnalyticsIUI
Prefer2SD: A Human-in-the-Loop Approach to Balancing Similarity and Diversity in In-Game Friend RecommendationsIn-game friend recommendations significantly impact player retention and sustained engagement in online games. Balancing similarity and diversity in recommendations is crucial for fostering stronger social bonds across diverse player groups. However, automated recommendation systems struggle to achieve this balance, especially as player preferences evolve over time. To tackle this challenge, we introduce Prefer2SD (derived from Preference to Similarity and Diversity), an iterative, human-in-the-loop approach designed to optimize the similarity-diversity (SD) ratio in friend recommendations. Developed in collaboration with a local game company, Prefer2D leverages a visual analytics system to help experts explore, analyze, and adjust friend recommendations dynamically, incorporating players' shifting preferences. The system employs interactive visualizations that enable experts to fine-tune the balance between similarity and diversity for distinct player groups. We demonstrate the efficacy of Prefer2SD through a within-subjects study (N=12), a case study, and expert interviews, showcasing its ability to enhance in-game friend recommendations and offering insights for the broader field of personalized recommendation systems.2025XWXiyuan Wang et al.Recommender System UXGamification DesignIUI
DancingBoard: Streamlining the Creation of Motion Comics to Enhance NarrativesMotion comics, a digital animation format that enhances comic book narratives, has wide applications in storytelling, education, and advertising. However, their creation poses significant challenging for amateur creators, primarily due to the need for specialized skills and complex workflows. To address these issues, we conducted an exploratory survey ($N=58$) to understand challenges associated with creating motion comics, and an expert interview ($N=4$) to identify a typical workflow for creation. We further analyzed $95$ online motion comics to gain insights into the design space of character and object actions. Based on our findings, we proposed \textit{DancingBoard}, an integrated authoring tool designed to simplify the creation process. This tool features a user-friendly interface and a guided workflow, providing comprehensive support throughout each step of the creation process. A user study involving $23$ creators showed that, comparing to professional tools, \textit{DancingBoard} is easily comprehensible and provides improved guidance and support, requiring less efforts from users. Additionally, a separate study with $18$ audience members confirmed the tool's effectiveness in conveying the story to its viewers.2025LCLongfei Chen et al.3D Modeling & AnimationCreative Coding & Computational ArtInteractive Narrative & Immersive StorytellingIUI
Bystander Privacy in Video Sharing Era: Automated Consent Compliance through Platform CensorshipBystander privacy has become a critical concern amidst the widespread activities of video sharing, engaging billions of users daily. Concerns arise when individuals inadvertently appear in public videos without consent. Existing methods for determining bystander permissions require significant adaptation and modifications by videographers and video sharing platforms, potentially limiting their adoption. This study explores leveraging platform censorship capabilities to enforce bystander privacy. We introduce selfFlag, a type of violative media signal designed to trigger automatic content flagging. Bystanders exhibiting such signals, captured in public videos, can be automatically identified and removed by platforms, thereby indirectly enforcing privacy preferences, primarily through the efforts of bystanders themselves. We conduct thorough measurements on current censorship practices, propose music-based triggering content, and develop an auxiliary tool for videographers to produce high-quality content with privacy compliance.2025SLSi LIAO et al.ShanghaiTech UniversityPrivacy by Design & User ControlPrivacy Perception & Decision-MakingCHI
AppAgent: Multimodal Agents as Smartphone UsersRecent advancements in large language models (LLMs) have led to the creation of intelligent agents capable of performing complex tasks. This paper introduces a novel LLM-based multimodal agent framework designed to operate smartphone applications. Our framework allows the agent to mimic human-like interactions such as tapping and swiping through a simplified action space, eliminating the need for system back-end access and enhancing its versatility across various apps. Central to the agent's functionality is an innovative in-context learning method, where it either autonomously explores or learns from human demonstrations, creating a knowledge base used to execute complex tasks across diverse applications. We conducted extensive testing with our agent on over 50 tasks spanning 10 applications, ranging from social media to sophisticated image editing tools. Additionally, a user study confirmed the agent's superior performance and practicality in handling a diverse array of high-level tasks, demonstrating its effectiveness in real-world settings. Our project page is available at \url{https://appagent-official.github.io/}.2025CZChi Zhang et al.Westlake University, School of EngineeringHuman-LLM CollaborationCHI
Advancing Problem-Based Learning with Clinical Reasoning for Improved Differential Diagnosis in Medical EducationMedical education increasingly emphasizes students' ability to apply knowledge in real-world clinical settings, focusing on evidence-based clinical reasoning and differential diagnoses. Problem-based learning (PBL) addresses traditional teaching limitations by embedding learning into meaningful contexts and promoting active participation. However, current PBL practices are often confined to medical instructional settings, limiting students' ability to self-direct and refine their approaches based on targeted improvements. Additionally, the unstructured nature of information organization during analysis poses challenges for record-keeping and subsequent review. Existing research enhances PBL realism and immersion but overlooks the construction of logic chains and evidence-based reasoning. To address these gaps, we designed e-MedLearn, a learner-centered PBL system that supports more efficient application and practice of evidence-based clinical reasoning. Through controlled study (N=19) and testing interviews (N=13), we gathered data to assess the system's impact. The findings demonstrate that e-MedLearn improves PBL experiences and provides valuable insights for advancing clinical reasoning-based learning.2025YXYuansong Xu et al.ShanghaiTech University, School of Information Science and TechnologyIntelligent Tutoring Systems & Learning AnalyticsSurgical Assistance & Medical TrainingCHI
CoKnowledge: Supporting Assimilation of Time-synced Collective Knowledge in Online Science VideosDanmaku, a system of scene-aligned, time-synced, floating comments, can augment video content to create `collective knowledge'. However, its chaotic nature often hinders viewers from effectively assimilating the collective knowledge, especially in knowledge-intensive science videos. With a formative study, we examined viewers' practices for processing collective knowledge and the specific barriers they encountered. Building on these insights, we designed a processing pipeline to filter, classify, and cluster danmaku, leading to the development of CoKnowledge -- a tool incorporating a video abstract, knowledge graphs, and supplementary danmaku features to support viewers' assimilation of collective knowledge in science videos. A within-subject study (N=24) showed that CoKnowledge significantly enhanced participants’ comprehension and recall of collective knowledge compared to a baseline with unprocessed live comments. Based on our analysis of user interaction patterns and feedback on design features, we presented design considerations for developing similar support tools.2025YZYuanhao Zhang et al.Hong Kong University of Science and Technology, CSEOnline Learning & MOOC PlatformsSTEM Education & Science CommunicationCHI
ClueCart: Supporting Game Story Interpretation and Narrative Inference from Fragmented CluesIndexical storytelling is gaining popularity in video games, where the narrative unfolds through fragmented clues. This approach fosters player-generated content and discussion, as story interpreters piece together the overarching narrative from these scattered elements. However, the fragmented and non-linear nature of the clues makes systematic categorization and interpretation challenging, potentially hindering efficient story reconstruction and creative engagement. To address these challenges, we first proposed a hierarchical taxonomy to categorize narrative clues, informed by a formative study. Using this taxonomy, we designed ClueCart, a creativity support tool aimed at enhancing creators' ability to organize story clues and facilitate intricate story interpretation. We evaluated ClueCart through a between-subjects study (N=40), using Miro as a baseline. The results showed that ClueCart significantly improved creators' efficiency in organizing and retrieving clues, thereby better supporting their creative processes. Additionally, we offer design insights for future studies focused on player-centric narrative analysis.2025XWXiyuan Wang et al.ShanghaiTech University, School of Information Science and TechnologyRole-Playing & Narrative GamesInteractive Narrative & Immersive StorytellingCHI
BPCoach: Exploring Hero Drafting in Professional MOBA Tournaments via Visual AnalyticsHero drafting for multiplayer online arena (MOBA) games is crucial because drafting directly affects the outcome of a match. Both sides take turns to "ban"/"pick" a hero from a roster of approximately 100 heroes to assemble their drafting. In professional tournaments, the process becomes more complex as teams are not allowed to pick heroes used in the previous rounds with the "best-of-N" rule. Additionally, human factors including the team's familiarity with drafting and play styles are overlooked by previous studies. Meanwhile, the huge impact of patch iteration on drafting strengths in the professional tournament is of concern. To this end, we propose a visual analytics system, BPCoach, to facilitate hero drafting planning by comparing various drafting through recommendations and predictions and distilling relevant human and in-game factors. Two case studies, expert feedback, and a user study suggest that BPCoach helps determine hero drafting in a rounded and efficient manner.2024SLShiyi Liu et al.Session 4f: Multiplayer Gaming and CommunicationCSCW
Designing Collaborative Technology for Intergenerational Social Play over DistanceCollaborative social play not only offers entertainment but also acts as a powerful catalyst in nurturing intergenerational relationships and strengthening community ties. In order to explore user preferences and interaction dynamics within this context, we investigated remote intergenerational communication through an analysis of 15 groups of parents and children from 16 families. Given the importance of collaborative activities in family and intergenerational connections, we aim to understand how these interactions and practices can adapt to remote environments. Our findings highlight both similarities and notable differences in the use of communication methods, workspace, and objects between parents and children. Particularly, we observed distinct patterns in gestural and verbal communication and identified specific challenges encountered by children in remote settings. Our findings also revealed the dynamics of play sessions, especially when co-located participants are involved, shedding light on the complexities of remote intergenerational communication. Our work offers valuable empirical insights on designing more effective and engaging remote collaborative platforms for families.2024YYYe Yuan et al.Session 4a: Intergenerational ConnectionsCSCW
PmTrack: Enabling Personalized mmWave-based Human TrackingLiu 等人提出 PmTrack 个性化毫米波人体追踪方案，通过用户特征建模实现精准追踪，提升人机交互与智能监控系统性能。2024HLHankai Liu et al.Human Pose & Activity RecognitionUbiComp
NotePlayer: Engaging Jupyter Notebooks for Dynamic Presentation of Analytical ProcessesDiverse presentation formats play a pivotal role in effectively conveying code and analytical processes during data analysis. One increasingly popular format is tutorial videos, particularly those based on Jupyter notebooks, which offer an intuitive interpretation of code and vivid explanations of analytical procedures. However, creating such videos requires a diverse skill set and significant manual effort, posing a barrier for many analysts. To bridge this gap, we introduce an innovative tool called NotePlayer, which connects notebook cells to video segments and incorporates a computational engine with language models to streamline video creation and editing. Our aim is to make the process more accessible and efficient for analysts. To inform the design of NotePlayer, we conducted a formative study and performed content analysis on a corpus of 38 Jupyter tutorial videos. This helped us identify key patterns and challenges encountered in existing tutorial videos, guiding the development of NotePlayer. Through a combination of a usage scenario and a user study, we validated the effectiveness of NotePlayer. The results show that the tool streamlines the video creation and facilitates the communication process for data analysts.2024YOYang Ouyang et al.Prototyping & User TestingComputational Methods in HCIUIST
An Interactive System for Suporting Creative Exploration of Cinematic Composition DesignsDesigning cinematic compositions, which involves moving cameras through a scene, is essential yet challenging in filmmaking. Machinima filmmaking provides real-time virtual environments for exploring different compositions flexibly and efficiently. However, producing high-quality cinematic compositions in such environments still requires significant cinematography skills and creativity. This paper presents Cinemassist, a tool designed to support and enhance this creative process by generating a variety of cinematic composition proposals at both keyframe and scene levels, which users can incorporate into their workflows and achieve more creative results. At the crux of our system is a deep generative model trained on real movie data, which can generate plausible, diverse camera poses conditioned on 3D animations and additional input semantics. Our model enables an interactive cinematic composition design workflow where users can co-design with the model by being inspired by model-generated suggestions while having control over the generation process. Our user study and expert rating find Cinemassist can facilitate the design process for users of different backgrounds and enhance the design quality especially for users with animation expertise, demonstrating its potential as an invaluable tool in the context of digital filmmaking.2024RHRui He et al.Generative AI (Text, Image, Music, Video)Video Production & Editing3D Modeling & AnimationUIST
BiasEye: A Bias-Aware Real-time Interactive Material Screening System for Impartial Candidate AssessmentIn the process of evaluating competencies for job or student recruitment through material screening, decision-makers can be influenced by inherent cognitive biases, such as the screening order or anchoring information, leading to inconsistent outcomes. To tackle this challenge, we conducted interviews with seven experts to understand their challenges and needs for support in the screening process. Building on their insights, we introduce BiasEye, a bias-aware real-time interactive material screening visualization system. BiasEye enhances awareness of cognitive biases by improving information accessibility and transparency. It also aids users in identifying and mitigating biases through a machine learning (ML) approach that models individual screening preferences. Findings from a mixed-design user study with 20 participants demonstrate that, compared to a baseline system lacking our bias-aware features, BiasEye increases participants' bias awareness and boosts their confidence in making final decisions. At last, we discuss the potential of ML and visualization in mitigating biases during human decision-making tasks.2024QLQianyu Liu et al.Explainable AI (XAI)Algorithmic Transparency & AuditabilityInteractive Data VisualizationIUI
Enabling Tangible Interaction on Non-touch Displays with Optical Mouse Sensor and Visible Light CommunicationThis paper presents Centaur, an input system that enables tangible interaction on displays, e.g., untouchable computer monitors. Centaur’s tangibles are built from low-cost optical mouse sensors, or can alternatively be emulated by commercial optical mice already available. They are trackable when put on the display, rendering a real-time and high-precision tangible interface. Even for ordinary personal computers, enabling Centaur requires no new hardware and installation burden. Centaur’s cost-effectiveness and wide availability open up new opportunities for tangible user interface (TUI) users and practitioners. Centaur’s key innovation lies in its tracking method. It embeds high-frequency light signals into different portions of the display content as location beacons. When the tangibles are put on the screen, they are able to sense the light signals with their optical mouse sensors, and thus determine the locations accordingly. We develop four applications to showcase the potential usage of Centaur.2022YYYihui Yan et al.ShanghaiTech UniversityCustomizable & Personalized ObjectsPrototyping & User TestingCHI
RoleSeer: Understanding Informal Social Role Changes in MMORPGs via Visual AnalyticsMassively multiplayer online role-playing games create virtual communities that support heterogeneous ``social roles'' determined by gameplay interaction behaviors under a specific social context. For all social roles, formal roles are pre-defined, obvious, and explicitly ascribed to the people holding the roles, whereas informal roles are not well-defined and unspoken. Identifying the informal roles and understanding their subtle changes are critical to designing sociability mechanisms. However, it is nontrivial to understand the existence and evolution of such roles due to their loosely defined, interconvertible, and dynamic characteristics. We propose a visual analytics system, RoleSeer, to investigate informal roles from the perspectives of behavioral interactions and depict their dynamic interconversions and transitions. Two cases, experts' feedback, and a user study suggest that RoleSeer helps interpret the identified informal roles and explore the patterns behind role changes. We see our approach's potential in investigating informal roles in a broader range of social games.2022LXLaixin Xie et al.ShanghaiTech UniversityGame UX & Player BehaviorRole-Playing & Narrative GamesMisinformation & Fact-CheckingCHI