Uncovering How Scatterplot Features Skew Visual Class SeparationMulti-class scatterplots are essential for visually comparing data, such as examining class distributions in dimensionality reduction and evaluating classification models. Visual class separation (VCS) measures quantify human perception but are largely derived from and evaluated with datasets reflecting limited types of scatterplot features (e.g., data distribution, similar class densities). Quantitatively identifying which scatterplot features are influential to VCS tasks can enable more robust guidance for future measures. We analyze the alignment between VCS measures and people's perceptions of class separation through a crowdsourced study using 70 scatterplot features relevant to class separation. To cover a wide range of scatterplot features, we generated a set of multi-class scatterplots from 6,947 real-world datasets. Our results highlight that multiple combinations of features are needed to best explain VCS. From our analysis, we develop a composite feature model that identifies key scatterplot features for measuring VCS task performance.2025SBS. Sandra Bae et al.University of Colorado Boulder, ATLAS InstituteInteractive Data VisualizationVisualization Perception & CognitionCHI
"Different and Boundary-Pushing:" How Blind and Low Vision Youth Live Code TogetherLive coding, or real-time algorithmic performance, is a rich medium for engaging novices in informal creative STEM learning. However, despite inclusive and open-source communities, disabled practitioners are underrepresented in live coding, and prior work highlights numerous accessibility barriers. To understand the perspectives of Blind and Low Vision (BLV) live coders, we formed FiLOrk (Fil Laptop Orchestra) with five BLV teens. Across two semesters, FiLOrk performed three original works, each guided by a core concept and improvisational structure for manipulating code and maintaining shared awareness. We interviewed four musicians to understand how they felt about the learning environment and how their creative identities formed individually and in relation to one another. We reflect on FiLOrk's outcomes and propose strategies for future live coding ensembles to meaningfully include novices with and without disabilities.2024WPWilliam Christopher Payne et al.Teleoperated DrivingUniversal & Inclusive DesignSpecial Education TechnologyC&C
PD-Insighter: A Visual Analytics System to Monitor Daily Actions for Parkinson's Disease TreatmentPeople with Parkinson's Disease (PD) can slow the progression of their symptoms with physical therapy. However, clinicians lack insight into patients’ motor function during daily life, preventing them from tailoring treatment protocols to patient needs. This paper introduces PD-Insighter, a system for comprehensive analysis of a person's daily movements for clinical review and decision-making. PD-Insighter provides an overview dashboard for discovering motor patterns and identifying critical deficits during activities of daily living and an immersive replay for closely studying the patient's body movements with environmental context. Developed using an iterative design study methodology in consultation with clinicians, we found that PD-Insighter's ability to aggregate and display data with respect to time, actions, and local environment enabled clinicians to assess a person's overall functioning during daily life outside the clinic. PD-Insighter's design offers future guidance for generalized multiperspective body motion analytics, which may significantly improve clinical decision-making and slow the functional decline of PD and other medical conditions.2024JKJade Kandel et al.University of North Carolina at Chapel HillHuman Pose & Activity RecognitionMedical & Scientific Data VisualizationTelemedicine & Remote Patient MonitoringCHI
Investigating the Mechanisms by which Prevalent Online Community Behaviors Influence Responses to Misinformation: Do Perceived Norms Really Act as a Mediator?This study addresses two currently open questions about how behaviors of online community members influence others' responses to misinformation. First, in contrast to prior work, it directly measures norm perception to address whether (1) norm perception actually acts as a mediator, (2) others' behaviors directly influence individuals' responses to misinformation, (3) both direct and mediated effects occur. Second, it investigates norm perceptions about a behavior that is not readily observable in online communities, but is prone to misinformation, specifically, vaccination. To do so, it experimentally manipulates the prevalence of communicating about vaccination (an unobservable behavior) within an online community. The results demonstrate no evidence of a direct effect---the causal relationship between prevalence of communicating a behavior and intentions to respond to misinformation only occurs via norm perception as a mediator. The paper highlights implications of these findings for designing community-centered interventions to influence perceived norms, thereby mitigating misinformation spread and impacts.2024ZAZhila Aghajari et al.Lehigh UniversityMisinformation & Fact-CheckingCommunity Engagement & Civic TechnologyCHI
Cieran: Designing Sequential Colormaps via In-Situ Active Preference LearningQuality colormaps can help communicate important data patterns. However, finding an aesthetically pleasing colormap that looks "just right" for a given scenario requires significant design and technical expertise. We introduce Cieran, a tool that allows any data analyst to rapidly find quality colormaps while designing charts within Jupyter Notebooks. Our system employs an active preference learning paradigm to rank expert-designed colormaps and create new ones from pairwise comparisons, allowing analysts who are novices in color design to tailor colormaps to their data context. We accomplish this by treating colormap design as a path planning problem through the CIELAB colorspace with a context-specific reward model. In an evaluation with twelve scientists, we found that Cieran effectively modeled user preferences to rank colormaps and leveraged this model to create new quality designs. Our work shows the potential of active preference learning for supporting efficient visualization design optimization.2024MHMatt-Heun Hong et al.University of North Carolina at Chapel HillInteractive Data VisualizationVisualization Perception & CognitionCHI
Usable News Authentication: How the Presentation and Location of Cryptographic Information Impacts the Usability of Provenance Information and Perceptions of News ArticlesCryptographic tools for authenticating the provenance of web-based information are a promising approach to increasing trust in online news and information. However, making these tools' technical assurances sufficiently usable for news consumers is essential to realizing their potential. We conduct an online study with 160 participants to investigate how the presentation (visual vs. textual) and location (on a news article page or a third-party site) of the provenance information affects news consumers' perception of the content's credibility and trustworthiness, as well as the usability of the tool itself. We find that although the visual presentation of provenance information is more challenging to adopt than its text-based counterpart, this approach leads its users to put more faith in the credibility and trustworthiness of digital news, especially when situated internally to the news article.2024EIErrol Francis II et al.Clemson UniversityAlgorithmic Transparency & AuditabilityPrivacy Perception & Decision-MakingMisinformation & Fact-CheckingCHI
Negotiating Sociotechnical Boundaries: Moderation Work to Counter Racist Attacks in Online CommunitiesOnline communities are susceptible to racist attacks, even when community policies explicitly prohibit racism. Drawing on the concept of symbolic boundary, we explored how community members sustained their communities against the perpetuation of racist logics and practices on Reddit. We drew on trace ethnography to analyze conversations about crime in two city subreddits (i.e., r/baltimore and r/chicago). The findings illustrate that the fragility of community boundaries was contributed by race baiting posts, covert racism, and racist brigading. At the same time, our research highlights that moderation efforts maintained and established institutional, cultural, and geographical boundaries to combat racist attacks. We discuss boundary as a design technique for building safe spaces for community members. Content warning: This work contains racist quotes that can upset or harm some readers.2024QWQunfang Wu et al.University of North Carolina at Chapel HillOnline Harassment & Counter-ToolsContent Moderation & Platform GovernanceGender & Race Issues in HCICHI
Do You See What I See? A Qualitative Study Eliciting High-Level Visualization ComprehensionDesigners often create visualizations to achieve specific high-level analytical or communication goals. These goals require people to naturally extract complex, contextualized, and interconnected patterns in data. While limited prior work has studied general high-level interpretation, prevailing perceptual studies of visualization effectiveness primarily focus on isolated, predefined, low-level tasks, such as estimating statistical quantities. This study more holistically explores visualization interpretation to examine the alignment between designers' communicative goals and what their audience sees in a visualization, which we refer to as their comprehension. We found that statistics people effectively estimate from visualizations in classical graphical perception studies may differ from the patterns people intuitively comprehend in a visualization. We conducted a qualitative study on three types of visualizations---line graphs, bar graphs, and scatterplots---to investigate the high-level patterns people naturally draw from a visualization. Participants described a series of graphs using natural language and think-aloud protocols. We found that comprehension varies with a range of factors, including graph complexity and data distribution. Specifically, 1) a visualization's stated objective often does not align with people's comprehension, 2) results from traditional experiments may not predict the knowledge people build with a graph, and 3) chart type alone is insufficient to predict the information people extract from a graph. Our study confirms the importance of defining visualization effectiveness from multiple perspectives to assess and inform visualization practices.2024GQGhulam Jilani Quadri et al.University of North CarolinaInteractive Data VisualizationVisualization Perception & CognitionCHI
The Cyber-Physical Control Room: A Mixed Reality Interface for Mobile Robot Teleoperation and Human-Robot TeamingIn this work, we present the design and evaluation of an immersive Cyber-Physical Control Room interface for remote mobile robots that provides users with both robot-egocentric and robot-exocentric 3D perspectives. We evaluate the Cyber-Physical Control room against a traditional robot interface in a mock disaster response scenario that features a mixed human-robot field team. In our evaluation, we found that the Cyber-Physical Control Room improved robot operator effectiveness by 28% while navigating a complex warehouse environment and performing a visual search. The Cyber-Physical Control Room also enhanced various aspects of human-robot teaming, including conversational engagement, the ability of a remote robot teleoperator to track their human partner in the field, and opinions of human teammate leadership qualities.2024MWDaniel Szafir et al.Mixed Reality WorkspacesContext-Aware ComputingHuman-Robot Collaboration (HRC)HRI
GRAFS: Graphical Faceted Search System to Support Conceptual Understanding in Exploratory SearchWhen people search for information about a new topic within large document collections, they implicitly construct a mental model of the unfamiliar information space to represent what they currently know and guide their exploration into the unknown. Building this mental model can be challenging as it requires not only finding relevant documents but also synthesizing important concepts and the relationships that connect those concepts both within and across documents. This article describes a novel interactive approach designed to help users construct a mental model of an unfamiliar information space during exploratory search. We propose a new semantic search system to organize and visualize important concepts and their relations for a set of search results. A user study (n=20) was conducted to compare the proposed approach against a baseline faceted search system on exploratory literature search tasks. Experimental results show that the proposed approach is more effective in helping users recognize relationships between key concepts, leading to a more sophisticated understanding of the search topic while maintaining similar functionality and usability as a faceted search system.2024MGMengtian Guo et al.Interactive Data VisualizationData StorytellingIUI
Assessing User Trust in Active Learning Systems: Insights from Query Policy and Uncertainty VisualizationActive learning systems have become increasingly popular for various applications in machine learning (ML), including medical imaging, environmental monitoring, and geospatial analysis. These systems rely on inputs dynamically queried from people to enhance classification. Ensuring appropriate analyst trust in these systems remains a significant obstacle, as analysts may over-rely or under-rely on the system. Common active learning (AL) strategies enhance classification models by asking an analyst to provide labels for data points with the highest degree of uncertainty. However, model-centric policies do not consider potential priming effects on the analyst and how they will affect people's trust in the system post-training. In this paper, we present an empirical study assessing how AL query policies and visualizations that enhance transparency in a classifier’s certainty influence trust in automated image classifiers. We found that query policy may significantly influence an analyst’s perception of the system’s capabilities, while the level of visual transparency into classifier certainty may influence an analyst’s ability to perform the classification task. Our study informs the design of interactive labeling systems to help mitigate the effects of over-reliance.2024ITIan Thomas et al.Explainable AI (XAI)Interactive Data VisualizationUncertainty VisualizationIUI
GlassMessaging: Towards Ubiquitous Messaging Using OHMDshttps://doi.org/10.1145/36109312023NJNuwan Janaka et al.Context-Aware ComputingUbiquitous ComputingUbiComp
"How Do You Quantify How Racist Something Is?": Color-Blind Moderation in Decentralized GovernanceVolunteer moderators serve as gatekeepers for problematic content, such as racism and other forms of hate speech, on digital platforms. Prior studies have reported volunteer moderators' diverse roles in different governance models, highlighting the tensions between moderators and other stakeholders (e.g., administrative teams and users). Building upon prior research, this paper focuses on how volunteer moderators moderate racist content and how a platform's governance influences these practices. To understand how moderators deal with racist content, we conducted in-depth interviews with 13 moderators from city subreddits on Reddit. We found that moderators heavily relied on AutoMod to regulate racist content and racist user accounts. However, content that was crafted through covert racism and ``color-blind'' racial frames was not addressed well. We attributed these challenges in moderating racist content to (1) moderators' concerns of power corruption, (2) arbitrary moderator team structures, and (3) evolving forms of covert racism. Our results demonstrate that decentralized governance on Reddit could not support local efforts to regulate color-blind racism. Finally, we discuss the conceptual and practical ways to disrupt color-blind moderation.2023QWQunfang Wu et al.Race and BiasCSCW
Transparency, Trust, and Security Needs for the Design of Digital News Authentication ToolsAmericans' trust in news is declining, and authenticity and transparency challenges in digital publishing contexts pose unique challenges to the ability to effectively gratify their information-seeking needs via online media. Cryptographic technologies and web-based provenance indicators have the potential to enhance the trustworthiness and transparency of digital communication, but better understandings of news consumers practices and needs are required to develop practical tools. Through a representative online survey of 400 digital news consumers and 19 follow-up interviews, we investigate how users authenticate and assign trust to news content, and identify specific needs pertaining to news transparency and authentication that could be met by digital news authentication tools. While many users currently rely on political ideology to assess news trustworthiness, we find that users of all political orientations see value in independent provenance and authentication tools for digital news.2023EIErrol Francis II et al.Security and TrustCSCW
A Probabilistic Model and Metrics for Estimating Perceived Accessibility of Desktop Applications in Keystroke-Based Non-Visual InteractionsPerceived accessibility of an application is a subjective measure of how well an individual with a particular disability, skills, and goals experiences the application via assistive technology. This paper first presents a study with 11 blind users to report how they perceive the accessibility of desktop applications while interacting via assistive technology such as screen readers and a keyboard. The study identifies the low navigational complexity of the user interface (UI) elements as the primary contributor to higher perceived accessibility of different applications. Informed by this study, we develop a probabilistic model that accounts for the number of user actions needed to navigate between any two arbitrary UI elements within an application. This model contributes to the area of computational interaction for non-visual interaction. Next, we derive three metrics from this model: complexity, coverage, and reachability, which reveal important statistical characteristics of an application indicative of its perceived accessibility. The proposed metrics are appropriate for comparing similar applications and can be fine-tuned for individual users to cater to their skills and goals. Finally, we present five use cases, demonstrating how blind users, application developers, and accessibility practitioners can benefit from our model and metrics.2023MIMd Touhidul Islam et al.Pennsylvania State UniversityVisual Impairment Technologies (Screen Readers, Tactile Graphics, Braille)Motor Impairment Assistive Input TechnologiesUniversal & Inclusive DesignCHI
Measuring Categorical Perception in Color-Coded ScatterplotsScatterplots commonly use color to encode categorical data. However, as datasets increase in size and complexity, the efficacy of these channels may vary. Designers lack insight into how robust different design choices are to variations in category numbers. This paper presents a crowdsourced experiment measuring how the number of categories and choice of color encodings used in multiclass scatterplots influences the viewers’ abilities to analyze data across classes. Participants estimated relative means in a series of scatterplots with 2 to 10 categories encoded using ten color palettes drawn from popular design tools. Our results show that the number of categories and color discriminability within a color palette notably impact people's perception of categorical data in scatterplots and that the judgments become harder as the number of categories grows. We examine existing palette design heuristics in light of our results to help designers make robust color choices informed by the parameters of their data.2023CTChin Tseng et al.University of North Carolina at Chapel HillInteractive Data VisualizationGeospatial & Map VisualizationVisualization Perception & CognitionCHI
The social embeddedness of peer production: A comparative qualitative analysis of three Indian language Wikipedia editionsWhy do some peer production projects do a better job at engaging potential contributors than others? We address this question by comparing three Indian language Wikipedias, namely, —Malayalam, Marathi, and Kannada. We found that although the three projects share goals, technological infrastructure, and a similar set of challenges, Malayalam Wikipedia's community engages language speakers in contributing at a much higher rate than the others. Drawing from a grounded theory analysis of interviews with 18 community participants from the three projects, we found that experience with participatory governance and free/open-source software in the Malayalam community supported high engagement of contributors. Counterintuitively, we found that financial resources intended to increase participation in the Marathi and Kannada communities hindered the growth of these communities. Our findings underscore the importance of social and cultural context in the trajectories of peer production communities.2022SKSejal Khatri et al.University of WashingtonCommunity Collaboration & WikipediaDeveloping Countries & HCI for Development (HCI4D)CHI
Modeling and Leveraging Analytic Focus During Exploratory Visual AnalysisVisual analytics systems enable highly interactive exploratory data analysis. Across a range of fields, these technologies have been successfully employed to help users learn from complex data. However, these same exploratory visualization techniques make it easy for users to discover spurious findings. This paper proposes new methods to monitor a user's analytic focus during visual analysis of structured datasets and use it to surface relevant articles that contextualize the visualized findings. Motivated by interactive analyses of electronic health data, this paper introduces a formal model of analytic focus, a computational approach to dynamically update the focus model at the time of user interaction, and a prototype application that leverages this model to surface relevant medical publications to users during visual analysis of a large corpus of medical records. Evaluation results with 24 users show that the modeling approach has high levels of accuracy and is able to surface highly relevant medical abstracts.2021ZZZhilan Zhou et al.University of North Carolina at Chapel HillInteractive Data VisualizationMedical & Scientific Data VisualizationCHI
The New Reality of Reproducibility: The Role of Data Work in Scientific ResearchAlthough reproducibility--the idea that a valid scientific experiment can be repeated with similar results--is integral to our understanding of good scientific practice, it has remained a difficult concept to define precisely. Across scientific disciplines, the increasing prevalence of large datasets, and the computational techniques necessary to manage and analyze those datasets, has prompted new ways of thinking about reproducibility. We present findings from a qualitative study of a NSF--funded two-week workshop developed to introduce an interdisciplinary group of domain scientists to data-management techniques for data-intensive computing, with a focus on reproducible science. Our findings suggest that the introduction of data-related activities promotes a new understanding of reproducibility as a mechanism for local knowledge transfer and collaboration, particularly as regards efficient software reuse.2020MFMelanie Feinberg et al.Data WorkCSCW
Wikipedia Edit-a-thons as Sites of Public PedagogyWikipedia edit-a-thon events provide a targeted approach to incorporating new knowledge into the online encyclopedia while also offering pathways toward new editor participation. Through the analysis of interviews with 13 edit-a-thon facilitators, however, we find motivations for running edit-a-thons extend far beyond adding content and editors. In this paper, we uncover how a range of personal and institutional values inspire these event facilitators toward fulfilling broader goals, including fostering information literacy and establishing community relationships outside of Wikipedia. Along with reporting motivations, values, and goals, we also describe strategies the facilitators adopt in their practice. Next, we discuss challenges faced by facilitators as they organize edit-a-thons. We situate our findings within two complementary theoretical lenses—information ecologies and public pedagogy—to guide our interpretation. Finally, we suggest new ways in which edit-a-thons, as well as similar peer production events and communities, can be understood, studied, and evaluated.2020LMLaura March et al.Collaboration: Creating and Writing TogetherCSCW