FuzzySeek: Multimodal Refinement of Imprecise Video Queries for Moment RetrievalRecent AI advances have made it possible to retrieve specific moments from long-form videos using natural language queries. However, existing systems can struggle to align retrieval results with user intent due to the lack of means for users to express their intents in simple natural language text. Moreover, there is limited support for helping users express or refine their intents interactively. We present FuzzySeek, a video moment retrieval interface that supports the expression and specification of imprecise or broad exploratory queries through multimodal interaction. FuzzySeek proposes three key components (1) Multimodality-blended text querying to improve expressivity, enabling users to directly anchor multimodal content within their textual queries, (2) Proactive Multimodal Guidance, which identifies imprecise/broad terms and phrases and surfaces targeted clarifications across modalities to improve query specificity and, (3) Query rollback to enable iterative back and forth exploration to enable direct or exploratory searches. Through a technical evaluation, multiple illustrative use cases and a user study with 11 participants, we show that FuzzySeek improves clarification efficiency, reduces cognitive load, and better supports video moment retrieval for imprecise queries compared to a baseline system without such support.2026AMAditi Mishra et al.Fujitsu Research of AmericaExploratory Search & Information SeekingKnowledge Graph & Semantic SearchPhysical-Digital Hybrid InteractionIUI
"I think this is fair": Uncovering the Complexities of Stakeholder Decision-Making in AI Fairness AssessmentAssessing fairness in artificial intelligence (AI) typically involves AI experts who select protected features, fairness metrics, and set fairness thresholds to assess outcome fairness. However, little is known about how stakeholders, particularly those affected by AI outcomes but lacking AI expertise, assess fairness. To address this gap, we conducted a qualitative study with 26 stakeholders without AI expertise, representing potential decision subjects in a credit rating scenario, to examine how they assess fairness when placed in the role of deciding on features with priority, metrics, and thresholds. We reveal that stakeholders' fairness decisions are more complex than typical AI expert practices: they considered features far beyond legally protected features, tailored metrics for specific contexts, set diverse yet stricter fairness thresholds, and even preferred designing customized fairness. Our results extend the understanding of how stakeholders can meaningfully contribute to AI fairness governance and mitigation, underscoring the importance of incorporating stakeholders' nuanced fairness judgments.2026LLLin Luo et al.University of GlasgowAI Ethics, Fairness & AccountabilityExplainable AI (XAI)Algorithmic Fairness & BiasCHI
DataSpeck: An AI-Driven Human-in-the-Loop System for Automating Transformations in Data Conversion WorkflowsIn data-driven systems, integrating disparate data sources becomes challenging when incoming data does not conform to the system's specifications. Despite advances in automated schema matching systems, data integration tasks involving complex semantic interrelationships still require users to manually identify and define transformations between datasets, which can be cognitively demanding and time-consuming. We present DataSpeck, an end-to-end system that automates the conversion of disparate data sources to fit any pre-existing data specification. DataSpeck employs an AI-driven human-in-the-loop design, using LLMs to analyze semantic relationships and generate step-by-step transformation pipelines autonomously, while only requesting user attention to resolve semantic ambiguities. In our technical evaluation, DataSpeck successfully automated ~86% of varied data transformations while generating interpretable strategies with confidence scores and targeted clarification requests. In a user study (N=12), participants completed data conversion tasks ~53% faster with significantly reduced cognitive load using DataSpeck compared to Microsoft Excel with Copilot.2026ARAdil Rahman et al.University of VirginiaHuman-LLM CollaborationExplainable AI (XAI)AI-Assisted Decision-Making & AutomationCHI
New Enactions of Expertise: Software Engineers’ Evaluation and Demonstration of Coding Expertise with AI Coding Assistants AI coding assistants are changing how software engineers engage in coding work. This shift raises a key question: does the changing of coding work also alter how software engineers evaluate and demonstrate coding expertise? We explore this question through a simulated live coding interview involving two software engineers, one as evaluator and the other as candidate, with AI tools allowed. Participants continued to rely on familiar criteria but adjusted the evidence they sought, as AI assistants both introduced new forms of demonstrating expertise and obscured some established workflows. The importance of these evolving enactions varied with evaluators’ emphasis on implementation versus planning. Lacking a clear link to expertise, heightened productivity expectations created additional tensions around these evolving enactions. We conclude by discussing how extended enactions can be supported through AI-focused tools and training, and how tensions between diminished enactions and productivity call for collaborative attention.2026YJYeonju Jang et al.Cornell UniversityHuman-LLM CollaborationAI-Assisted Decision-Making & AutomationGenerative AI (Text, Image, Music, Video)CHI
EARN Fairness: Explaining, Asking, Reviewing, and Negotiating Artificial Intelligence Fairness Metrics Among StakeholdersNumerous fairness metrics have been proposed and employed by artificial intelligence (AI) experts to quantitatively measure bias and define fairness in AI models. Recognizing the need to accommodate stakeholders' diverse fairness understandings, efforts are underway to solicit their input. However, conveying AI fairness metrics to stakeholders without AI expertise, capturing their personal preferences, and seeking a collective consensus remain challenging and underexplored. To bridge this gap, we propose a new framework, EARN (Explain, Ask, Review, and Negotiate) Fairness, which facilitates collective metric decisions among stakeholders without requiring AI expertise. The framework features an adaptable interactive system and a stakeholder-centered EARN Fairness process to Explain fairness metrics, Ask stakeholders' personal metric preferences, Review metrics collectively, and Negotiate a consensus on metric selection. To gather empirical results, we applied the framework to a credit rating scenario and conducted a user study involving 18 decision subjects without AI knowledge. We elicited their personal metric preferences and subsequently we studied how they reached metric consensus in team sessions. Our work shows that the EARN Fairness framework supports stakeholders to express and negotiate fairness preferences, and we provide practical guidance for implementing human-centered AI fairness in high-risk contexts. Through this approach, we aim to reach consensus of fairness perspectives, fostering more equitable and inclusive AI fairness.2025LLLin Luo et al.Facilitating Equity and Fairness in TechCSCW
WhatIF: Branched Narrative Fiction Visualization for Authoring Emergent Narratives using Large Language ModelsBranched Narrative Fiction (BNF) are non-linear, text based narrative games, where the player of the game is an active participant shaping the story. Unlike linear narratives, BNF allows players to influence the direction, outcomes, and progression of the plot. A narrative fiction developer designs these branching storylines, creating a dynamic interaction between the player and the narrative which requires significant time and skill. In this work we build and investigate the use of a visual analytics tool to help narrative fiction developers generate and plan these parallel worlds within a BNF. We present WhatIF, a visual analytics tool that aids BNF developers to create BNF graphs, edit the graphs, obtain recommendations, visualize differences between storylines and finally verify their BNF on custom metrics. Through a formative study (3 participants) and a user study (11 participants), we observe that WhatIF helps users plan and prototype their BNF, provides avenues to support iterative refinement of narrative and also aids in removing writer's block. Furthermore, we explore how contemporary generative AI (GenAI) tools can empower game developers to build richer and more immersive narratives.2025AMAditi Mishra et al.Generative AI (Text, Image, Music, Video)AI-Assisted Creative WritingC&C
Kaleidoscope Gallery: Exploring Ethics and Generative AI Through ArtEthical theories and Generative AI (GenAI) models are dynamic concepts subject to continuous evolution. This paper investigates the visualization of ethics through a subset of GenAI models. We expand on the emerging field of Visual Ethics, using art as a form of critical inquiry and the metaphor of a kaleidoscope to invoke moral imagination. Through formative interviews with 10 ethics experts, we first establish a foundation of ethical theories. Our analysis reveals five families of ethical theories, which we then transform into images using the text-to-image (T2I) GenAI model. The resulting imagery, curated as Kaleidoscope Gallery and evaluated by the same experts, revealed eight themes that highlight how morality, society, and learned associations are central to ethical theories. We discuss implications for critically examining T2I models and present cautions and considerations. This work contributes to examining ethical theories as foundational knowledge that interrogates GenAI models as socio-technical systems.2025AIAlayt Issak et al.Generative AI (Text, Image, Music, Video)Explainable AI (XAI)Digital Art Installations & Interactive PerformanceC&C
AdaptiveSliders: User-aligned Semantic Slider-based Editing of Text-to-Image Model OutputPrecise editing of text-to-image model outputs remains challenging. Slider-based editing is a recent approach wherein the image’s semantic attributes are manipulated via sliders. However, it has significant user-centric issues. First, slider variations are often inconsistent across the sliding range. Second, the optimal slider range is unpredictable, with default values often being too large or small depending on the prompt and attribute. Third, manipulating one attribute can unintentionally alter others due to the complex entanglement of latent spaces. We introduce AdaptiveSliders, a tool that addresses these challenges by adapting to the specific attributes and prompts, generating consistent slider variations and optimal bounds while minimizing unintended changes. AdaptiveSliders also suggests initial attributes and generates initial images more aligned with prompt semantics. Through three validation studies and one end-to-end user study, we demonstrate that AdaptiveSliders significantly improves user control and experience, enabling semantic slider-based editing aligned with user needs and expectations.2025RJRahul Jain et al.Purdue University, Department of Electrical and Computer EngineeringGenerative AI (Text, Image, Music, Video)Explainable AI (XAI)CHI
Creative ML Assemblages: The interactive politics of people, processes, and productsCreative ML tools are collaborative systems that afford artistic creativity through their myriad interactive relationships. We propose using ``assemblage thinking" to support analyses of creative ML by approaching it as a system in which the elements of people, organizations, culture, practices, and technology constantly influence each other. We model these interactions as ``coordinating elements" that give rise to the social and political characteristics of a particular creative ML context, and call attention to three dynamic elements of creative ML whose interactions provide unique context for the social impact a particular system as: people, creative processes, and products. As creative assemblages are highly contextual, we present these as analytical concepts that computing researchers can adapt to better understand the functioning of a particular system or phenomena and identify intervention points to foster desired change. This paper contributes to theorizing interactions with AI in the context of art, and how these interactions shape the production of algorithmic art.2024RSRenee Shelby et al.Session 3a: AI in Creative Workflows: Opportunities and ChallengesCSCW
GO-Finder: A Registration-Free Wearable System for Assisting Users in Finding Lost Objects via Hand-Held Object DiscoveryPeople spend an enormous amount of time and effort looking for lost objects. Various computational systems have been developed to help remind people of the location of lost objects by providing information on their locations. However, prior systems for assisting people in finding objects require users to register the target objects in advance. This requirement imposes a cumbersome burden on the users, and the system cannot help remind them of unexpectedly lost objects. We propose a registration-free wearable camera based system for assisting people in finding an arbitrary number of objects based on two key features: automatic discovery of hand-held objects and image-based candidate selection. Given a video taken from a wearable camera, our system automatically detects and groups hand-held objects to form a visual timeline of the objects. Users can retrieve the last appearance of the object by browsing the timeline through a smartphone app. We conducted a user study to investigate how users benefit from using our system. We confirmed improved accuracy and reduced mental load regarding the object search task by providing clear visual cues on object locations.2021TYTakuma Yagi et al.Smartwatches & Fitness BandsContext-Aware ComputingIUI
Guided Play: Digital Sensing and Coaching for Stereotypical Play Behavior in Children with AutismRestricted and repetitive behaviors (RRBs) are a core symptom and an early marker of autism. Current research and intervention for RRB heavily rely on professional experience and effort. Guided Play is a technology that uses instrumented games and toys as a platform to understand children's play behavior and facilitate behavioral intervention during play. This paper presents the design and implementation of a prototype based on the technology, as well as an evaluation on 6 children with autism. The results show that children with RRBs in physical world activities also exhibit similar patterns in a similar digital activity, and that digital coaching can reduce RRBs by expanding children's play skill repertoire and promoting symbolic play.2019CCCong Chen et al.Cognitive Impairment & Neurodiversity (Autism, ADHD, Dyslexia)Special Education TechnologyIUI