Challenges in Synchronous & Remote Collaboration Around VisualizationWe characterize 16 challenges faced by those investigating and developing remote and synchronous collaborative experiences around visualization. Our work reflects the perspectives and prior research efforts of an international group of 29 experts from across human-computer interaction and visualization sub-communities. The challenges are anchored around five collaborative activities that exhibit a centrality of visualization and multimodal communication. These activities include exploratory data analysis, creative ideation, visualization-rich presentations, joint decision making grounded in data, and real-time data monitoring. The challenges also reflect the changing dynamics of these activities in the face of recent advances in extended reality (XR) and artificial intelligence (AI). As an organizing scheme for future research at the intersection of visualization and computer-supported cooperative work, we align the challenges with a sequence of four sets of research and development activities: technological choices, social factors, AI assistance, and evaluation.2026MBMatthew Brehmer et al.University of WaterlooInteractive Data VisualizationRemote Work Tools & ExperienceMulti-User Large Display CollaborationCHI
Empower Real-World BCIs with NIRS-X: An Adaptive Learning Framework that Harnesses Unlabeled Brain SignalsBrain-Computer Interfaces (BCIs) using functional near-infrared spectroscopy (fNIRS) hold promise for future interactive user interfaces due to their ease of deployment and declining cost. However, they typically require a separate calibration process for each user and task, which can be burdensome. Machine learning helps, but faces a data scarcity problem. Due to inherent inter-user variations in physiological data, it has been typical to create a new annotated training dataset for every new task and user. To reduce dependence on such extensive data collection and labeling, we present an adaptive learning framework, NIRS-X, to harness more easily accessible unlabeled fNIRS data. NIRS-X includes two key components: NIRSiam and NIRSformer. We use the NIRSiam algorithm to extract generalized brain activity representations from unlabeled fNIRS data obtained from previous users and tasks, and then transfer that knowledge to new users and tasks. In conjunction, we design a neural network, NIRSformer, tailored for capturing both local and global, spatial and temporal relationships in multi-channel fNIRS brain input signals. By using unlabeled data from both a previously released fNIRS2MW visual $n$-back dataset and a newly collected fNIRS2MW audio $n$-back dataset, NIRS-X demonstrates its strong adaptation capability to new users and tasks. Results show comparable or superior performance to supervised methods, making NIRS-X promising for real-world fNIRS-based BCIs.2024LWLiang Wang et al.Human Pose & Activity RecognitionBrain-Computer Interface (BCI) & NeurofeedbackUIST
To Live in Their Utopia: Why Algorithmic Systems Create Absurd OutcomesThe promise AI's proponents have made for decades is one in which our needs are predicted, anticipated, and met - often before we even realize it. Instead, algorithmic systems, particularly AIs trained on large datasets and deployed to massive scales, seem to keep making the wrong decisions, causing harm and rewarding absurd outcomes. Attempts to make sense of why AIs make wrong calls in the moment explain the instances of errors, but how the environment surrounding these systems precipitate those instances remains murky. This paper draws from anthropological work on bureaucracies, states, and power, translating these ideas into a theory describing the structural tendency for powerful algorithmic systems to cause tremendous harm. I show how administrative models and projections of the world create marginalization, just as algorithmic models cause representational and allocative harm. This paper concludes with a recommendation to avoid the absurdity algorithmic systems produce by denying them power.2021AAAli AlkhatibUniversity of San FranciscoAI Ethics, Fairness & AccountabilityAlgorithmic Fairness & BiasTechnology Ethics & Critical HCICHI
Human-in-the-Loop Machine Learning to Increase Video Accessibility for Visually Impaired and Blind UsersVideo accessibility is crucial for blind and visually impaired individuals for education, employment, and entertainment purposes. However, professional video descriptions are costly and time-consuming. Volunteer-created video descriptions could be a promising alternative, however, they can vary in quality and can be intimidating for novice describers. We developed a Human-in-the-Loop Machine Learning (HILML) approach to video description by automating video text generation and scene segmentation and allowing humans to edit the output. The HILML approach facilitates human-machine collaboration to produce high quality video descriptions while keeping a low barrier to entry for volunteer describers. Our HILML system was significantly faster and easier to use for first-time video describers compared to a human-only control condition with no machine learning assistance. The quality of the video descriptions and understanding of the topic created by the HILML system compared to the human-only condition were rated as being significantly higher by blind and visually impaired users.2020BYBeste F Yuksel et al.Generative AI (Text, Image, Music, Video)Human-LLM CollaborationVisual Impairment Technologies (Screen Readers, Tactile Graphics, Braille)DIS
Investigating Implicit Gender Bias and Embodiment of White Males in Virtual Reality with Full Body Visuomotor SynchronyPrevious research has shown that when White people embody a black avatar in virtual reality (VR) with full body visuomotor synchrony, this can reduce their implicit racial bias. In this paper, we put men in female and male avatars in VR with full visuomotor synchrony using wearable trackers and investigated implicit gender bias and embodiment. We found that participants embodied in female avatars displayed significantly higher levels of implicit gender bias than those embodied in male avatars. The implicit gender bias actually increased after exposure to female embodiment in contrast to male embodiment. Results also showed that participants felt embodied in their avatars regardless of gender matching, demonstrating that wearable trackers can be used for a realistic sense of avatar embodiment in VR. We discuss the future implications of these findings for both VR scenarios and embodiment technologies.2019SLSarah Lopez et al.University of San FranciscoImmersion & Presence ResearchIdentity & Avatars in XRCHI
Designing Technology for an Aging PopulationThis course describes age-related factors that affect older adults’ ability to use digital technology, and present design guidelines that reflect older adults’ varied capabilities, usage patterns, and preferences. The course also describes the value of conducting usability tests with older adults, and how to do that successfully.2018JJJeff A JohnsonUniversity of San Francisco, IncAging-Friendly Technology DesignUniversal & Inclusive DesignCHI
Designing with the Mind in Mind: The Psychological Basis for UI Design GuidelinesUI design rules and guidelines are not simple recipes. Applying them effectively requires determining rule applicability and precedence and balancing trade-offs when rules compete. By understanding the underlying psychology, designers and evaluators enhance their ability to apply design rules. This two-part (160-minute) course explains that psychology.2018JJJeff A JohnsonUniversity of San FranciscoKnowledge Worker Tools & WorkflowsUser Research Methods (Interviews, Surveys, Observation)CHI
Brain-Computer Interfaces for Artistic ExpressionArtists have been using BCIs for artistic expression since the 1960s. Their interest and creativity is now increasing because of the availability of affordable BCI devices and software that does not require them to invest extensive time in getting the BCI to work or tuning it to their application. Designers of artistic BCIs are often ahead of more traditional BCI researchers in ideas on using BCIs in multimodal and multiparty contexts, where multiple users are involved, and where robustness and efficiency are not the main matters of concern. The aim of this workshop is to look at current (research) activities in BCIs for artistic expression and to identify research areas that are of interest for both BCI and HCI researchers as well as artists/designers of BCI applications.2018ANAnton Nijholt et al.University of TwenteBrain-Computer Interface (BCI) & NeurofeedbackDigital Art Installations & Interactive PerformanceCHI
Using Animation to Alleviate Overdraw in Multiclass Scatterplot MatricesThe scatterplot matrix (SPLOM) is a commonly used technique for visualizing multiclass multivariate data. However, multiclass SPLOMs have issues with overdraw (overlapping points), and most existing techniques for alleviating overdraw focus on individual scatterplots with a single class. This paper explores whether animation using flickering points is an effective way to alleviate overdraw in these multiclass SPLOMs. In a user study with 69 participants, we found that users not only performed better at identifying dense regions using animated SPLOMs, but also found them easier to interpret and preferred them to static SPLOMs. These results open up new directions for future work on alleviating overdraw for multiclass SPLOMs, and provide insights for applying animation to alleviate overdraw in other settings.2018HCHelen Chen et al.University of San FranciscoInteractive Data VisualizationVisualization Perception & CognitionCHI
Designing with the Mind in Mind: The Psychological Basis for UI Design GuidelinesUI design rules and guidelines are not simple recipes. Applying them effectively requires determining rule applicability and precedence and balancing trade-offs when rules compete. By understanding the underlying psychology, designers and evaluators enhance their ability to apply design rules. This two-part (160-minute) course explains that psychology.2018JJJeff A JohnsonUniversity of San FranciscoUser Research Methods (Interviews, Surveys, Observation)Prototyping & User TestingCHI