The Anatomy of a Plea: How Uncertainty, Visualizations & Individual Differences Shape Plea Bargain DecisionsPlea bargains are commonly used in the criminal justice system, where they can offer potential benefits to both the prosecution and the defendant. However, research has shown that defendants often engage in poor decision-making, such as accepting the plea even when the trial sentence is likely to be less severe. While previous studies have shown some evidence that uncertainty visualizations can improve decision-making, there is a lack of research on their effectiveness in domain-specific tasks like plea bargain decision-making. In this work, we conduct a series of experiments to explore whether the presence and format of uncertainty impact plea bargain decisions, taking into account time pressure and individual differences. Our findings reveal that these factors can have a significant impact on plea bargain decisions. We also show evidence that communicating uncertainty in the form of text can elicit more optimal decisions under time-pressure conditions.2025MBMelanie Bancilhon et al.US Army Research LaboratoryUncertainty VisualizationPrivacy Perception & Decision-MakingCHI
From Camera-Eye to AI: Exploring the Interplay of Cinematography and Computational Visual StorytellingWhile much prior work on computational visual storytelling analyzes image content, it largely overlooks formal elements. This raises the question: how might particular cinematographic techniques shape a system's interpretation and narration of imagery? To investigate this question, we generate 60 responses from a Vision Language Model using a multi-faceted prompt paired with different still frames from Man with a Movie Camera (1929), a silent documentary film renowned for its innovative cinematography. We present three themes that highlight roles of cinematography in computational visual storytelling: (1) how AI discerns drama and power from camera shots and angles that portray social reality; (2) how AI (mis)interprets lighting and focus techniques that compose ambiguous reality; and (3) how AI navigates visual effects that render surreality. In turn, we look toward cinematic controls to reimagine users as directors of visual storytelling systems and discuss how expressive AI can support speculating about the past.2025BHBrett A. Halperin et al.University of Washington, Human Centered Design & EngineeringGenerative AI (Text, Image, Music, Video)AI-Assisted Creative WritingVideo Production & EditingCHI
Artificial Dreams: Surreal Visual Storytelling as Inquiry Into AI 'Hallucination'What does it mean for stochastic artificial intelligence (AI) to “hallucinate” when performing a literary task as open-ended as creative visual storytelling? In this paper, we investigate AI “hallucination” by stress-testing a visual storytelling algorithm with different visual and textual inputs designed to probe dream logic inspired by cinematic surrealism. Following a close reading of 100 visual stories that we deem artificial dreams, we describe how AI “hallucination” in computational visual storytelling is the opposite of groundedness: literary expression that is ungrounded in the visual or textual inputs. We find that this lack of grounding can be a source of either creativity or harm entangled with bias and illusion. In turn, we disentangle these obscurities and discuss steps toward addressing the perils while harnessing the potentials for innocuous cases of AI “hallucination” to enhance the creativity of visual storytelling.2024BHBrett A. Halperin et al.Generative AI (Text, Image, Music, Video)Interactive Narrative & Immersive StorytellingDIS
Towards an Eye-Brain-Computer Interface: Combining Gaze with the Stimulus-Preceding Negativity for Target Selections in XRGaze-assisted interaction techniques enable intuitive selections without requiring manual pointing but can result in unintended selections, known as Midas touch. A confirmation trigger eliminates this issue but requires additional physical and conscious user effort. Brain-computer interfaces (BCIs), particularly passive BCIs harnessing anticipatory potentials such as the Stimulus-Preceding Negativity (SPN) - evoked when users anticipate a forthcoming stimulus - present an effortless implicit solution for selection confirmation. Within a VR context, our research uniquely demonstrates that SPN has the potential to decode intent towards the visually focused target. We reinforce the scientific understanding of its mechanism by addressing a confounding factor - we demonstrate that the SPN is driven by the user's intent to select the target, not by the stimulus feedback itself. Furthermore, we examine the effect of familiarly placed targets, finding that SPN may be evoked quicker as users acclimatize to target locations; a key insight for everyday BCIs.2024GRG S Rajshekar Reddy et al.University of Colorado BoulderEye Tracking & Gaze InteractionBrain-Computer Interface (BCI) & NeurofeedbackSocial & Collaborative VRCHI
XAutoML: A Visual Analytics Tool for Understanding and Validating Automated Machine LearningIn the last 10 years, various automated machine learning (AutoML) systems have been proposed to build end-to-end machine learning (ML) pipelines with minimal human interaction. Even though such automatically synthesized ML pipelines are able to achieve competitive performance, recent studies have shown that users do not trust models constructed by AutoML due to missing transparency of AutoML systems and missing explanations for the constructed ML pipelines. In a requirements analysis study with 36 domain experts, data scientists, and AutoML researchers from different professions with vastly different expertise in ML, we collect detailed informational needs for AutoML. We propose XAutoML, an interactive visual analytics tool for explaining arbitrary AutoML optimization procedures and ML pipelines constructed by AutoML. XAutoML combines interactive visualizations with established techniques from explainable artificial intelligence (XAI) to make the complete AutoML procedure transparent and explainable. By integrating XAutoML with JupyterLab, experienced users can extend the visual analytics with ad-hoc visualizations based on information extracted from XAutoML. We validate our approach in a user study with the same diverse user group from the requirements analysis. All participants were able to extract useful information from XAutoML, leading to a significantly increased understanding of ML pipelines produced by AutoML and the AutoML optimization itself.2024MZMarc-Andr? Z?ller et al.Explainable AI (XAI)AutoML InterfacesInteractive Data VisualizationIUI
Envisioning Narrative Intelligence: A Creative Visual Storytelling AnthologyIn this paper, we collect an anthology of 100 visual stories from authors who participated in our systematic creative process of improvised story-building based on image sequences. Following close reading and thematic analysis of our anthology, we present five themes that characterize the variations found in this creative visual storytelling process: (1) Narrating What is in Vision vs. Envisioning; (2) Dynamically Characterizing Entities/Objects; (3) Sensing Experiential Information About the Scenery; (4) Modulating the Mood; (5) Encoding Narrative Biases. In understanding the varied ways that people derive stories from images, we offer considerations for collecting story-driven training data to inform automatic story generation. In correspondence with each theme, we envision narrative intelligence criteria for computational visual storytelling as: creative, reliable, expressive, grounded, and responsible. From these criteria, we discuss how to foreground creative expression, account for biases, and operate in the bounds of visual storyworlds.2023BHBrett A. Halperin et al.University of WashingtonGenerative AI (Text, Image, Music, Video)Interactive Narrative & Immersive StorytellingCHI
Personalized Explanations for Hybrid Recommender SystemsRecommender systems have become pervasive on the web, shaping the way users see information and thus the decisions they make. As these systems get more complex, there is a growing need for transparency. In this paper, we study the problem of generating and visualizing personalized explanations for hybrid recommender systems, which incorporate many different data sources. We build upon a hybrid probabilistic graphical model and develop an approach to generate real-time recommendations along with personalized explanations. To study the benefits of explanations for hybrid recommender systems, we conduct a crowd-sourced user study where our system generates personalized recommendations and explanations for real users of the last.fm music platform. We experiment with 1) different explanation styles (e.g., user-based, item-based), 2) manipulating the number of explanation styles presented, and 3) manipulating the presentation format (e.g., textual vs. visual). We apply a mixed model statistical analysis to consider user personality traits as a control variable and demonstrate the usefulness of our approach in creating personalized hybrid explanations with different style, number, and format.2019PKPigi Kouki et al.Explainable AI (XAI)Recommender System UXData StorytellingIUI
I Can Do Better Than Your AI: Expertise and ExplanationsIntelligent assistants, such as navigation, recommender, and expert systems, are most helpful in situations where users lack domain knowledge. Despite this, recent research in cognitive psychology has revealed that lower-skilled individuals may maintain a sense of illusory superiority, which might suggest that users with the highest need for advice may be the least likely to defer judgment. Explanation interfaces -- a method for persuading users to take a system's advice -- are thought by many to be the solution for instilling trust, but do their effects hold for self-assured users? To address this knowledge gap, we conducted a quantitative study (N=529) wherein participants played a binary decision-making game with help from an intelligent assistant. Participants were profiled in terms of both actual (measured) expertise and reported familiarity with the task concept. The presence of explanations, level of automation, and number of errors made by the intelligent assistant were manipulated while observing changes in user acceptance of advice. An analysis of cognitive metrics lead to three findings for research in intelligent assistants: 1) higher reported familiarity with the task simultaneously predicted more reported trust but less adherence, 2) explanations only swayed people who reported very low task familiarity, and 3) showing explanations to people who reported more task familiarity led to automation bias.2019JSJames Schaffer et al.Explainable AI (XAI)AI-Assisted Decision-Making & AutomationIUI