Empower Real-World BCIs with NIRS-X: An Adaptive Learning Framework that Harnesses Unlabeled Brain SignalsBrain-Computer Interfaces (BCIs) using functional near-infrared spectroscopy (fNIRS) hold promise for future interactive user interfaces due to their ease of deployment and declining cost. However, they typically require a separate calibration process for each user and task, which can be burdensome. Machine learning helps, but faces a data scarcity problem. Due to inherent inter-user variations in physiological data, it has been typical to create a new annotated training dataset for every new task and user. To reduce dependence on such extensive data collection and labeling, we present an adaptive learning framework, NIRS-X, to harness more easily accessible unlabeled fNIRS data. NIRS-X includes two key components: NIRSiam and NIRSformer. We use the NIRSiam algorithm to extract generalized brain activity representations from unlabeled fNIRS data obtained from previous users and tasks, and then transfer that knowledge to new users and tasks. In conjunction, we design a neural network, NIRSformer, tailored for capturing both local and global, spatial and temporal relationships in multi-channel fNIRS brain input signals. By using unlabeled data from both a previously released fNIRS2MW visual $n$-back dataset and a newly collected fNIRS2MW audio $n$-back dataset, NIRS-X demonstrates its strong adaptation capability to new users and tasks. Results show comparable or superior performance to supervised methods, making NIRS-X promising for real-world fNIRS-based BCIs.2024LWLiang Wang et al.Human Pose & Activity RecognitionBrain-Computer Interface (BCI) & NeurofeedbackUIST
De-Stijl: Facilitating Graphics Design with Interactive 2D Color Palette RecommendationSelecting a proper color palette is critical in crafting a high-quality graphic design to gain visibility and communicate ideas effectively. To facilitate this process, we propose De-Stijl, an intelligent and interactive color authoring tool to assist novice designers in crafting harmonic color palettes, achieving quick design iterations, and fulfilling design constraints. Through De-Stijl, we contribute a novel 2D color palette concept that allows users to intuitively perceive color designs in context with their proportions and proximities. Further, De-Stijl implements a holistic color authoring system that supports 2D palette extraction, theme-aware and spatial-sensitive color recommendation, and automatic graphical elements (re)colorization. We evaluated De-Stijl through an in-lab user study by comparing the system with existing industry standard tools, followed by in-depth user interviews. Quantitative and qualitative results demonstrate that De-Stijl is effective in assisting novice design practitioners to quickly colorize graphic designs and easily deliver several alternatives.2023XSAlark Joshi et al.University of Waterloo360° Video & Panoramic ContentGraphic Design & Typography ToolsPrototyping & User TestingCHI
To Live in Their Utopia: Why Algorithmic Systems Create Absurd OutcomesThe promise AI's proponents have made for decades is one in which our needs are predicted, anticipated, and met - often before we even realize it. Instead, algorithmic systems, particularly AIs trained on large datasets and deployed to massive scales, seem to keep making the wrong decisions, causing harm and rewarding absurd outcomes. Attempts to make sense of why AIs make wrong calls in the moment explain the instances of errors, but how the environment surrounding these systems precipitate those instances remains murky. This paper draws from anthropological work on bureaucracies, states, and power, translating these ideas into a theory describing the structural tendency for powerful algorithmic systems to cause tremendous harm. I show how administrative models and projections of the world create marginalization, just as algorithmic models cause representational and allocative harm. This paper concludes with a recommendation to avoid the absurdity algorithmic systems produce by denying them power.2021AAAli AlkhatibUniversity of San FranciscoAI Ethics, Fairness & AccountabilityAlgorithmic Fairness & BiasTechnology Ethics & Critical HCICHI
Human-in-the-Loop Machine Learning to Increase Video Accessibility for Visually Impaired and Blind UsersVideo accessibility is crucial for blind and visually impaired individuals for education, employment, and entertainment purposes. However, professional video descriptions are costly and time-consuming. Volunteer-created video descriptions could be a promising alternative, however, they can vary in quality and can be intimidating for novice describers. We developed a Human-in-the-Loop Machine Learning (HILML) approach to video description by automating video text generation and scene segmentation and allowing humans to edit the output. The HILML approach facilitates human-machine collaboration to produce high quality video descriptions while keeping a low barrier to entry for volunteer describers. Our HILML system was significantly faster and easier to use for first-time video describers compared to a human-only control condition with no machine learning assistance. The quality of the video descriptions and understanding of the topic created by the HILML system compared to the human-only condition were rated as being significantly higher by blind and visually impaired users.2020BYBeste F Yuksel et al.Generative AI (Text, Image, Music, Video)Human-LLM CollaborationVisual Impairment Technologies (Screen Readers, Tactile Graphics, Braille)DIS
Investigating Implicit Gender Bias and Embodiment of White Males in Virtual Reality with Full Body Visuomotor SynchronyPrevious research has shown that when White people embody a black avatar in virtual reality (VR) with full body visuomotor synchrony, this can reduce their implicit racial bias. In this paper, we put men in female and male avatars in VR with full visuomotor synchrony using wearable trackers and investigated implicit gender bias and embodiment. We found that participants embodied in female avatars displayed significantly higher levels of implicit gender bias than those embodied in male avatars. The implicit gender bias actually increased after exposure to female embodiment in contrast to male embodiment. Results also showed that participants felt embodied in their avatars regardless of gender matching, demonstrating that wearable trackers can be used for a realistic sense of avatar embodiment in VR. We discuss the future implications of these findings for both VR scenarios and embodiment technologies.2019SLSarah Lopez et al.University of San FranciscoImmersion & Presence ResearchIdentity & Avatars in XRCHI
Designing Technology for an Aging PopulationThis course describes age-related factors that affect older adults’ ability to use digital technology, and present design guidelines that reflect older adults’ varied capabilities, usage patterns, and preferences. The course also describes the value of conducting usability tests with older adults, and how to do that successfully.2018JJJeff A JohnsonUniversity of San Francisco, IncAging-Friendly Technology DesignUniversal & Inclusive DesignCHI
Designing with the Mind in Mind: The Psychological Basis for UI Design GuidelinesUI design rules and guidelines are not simple recipes. Applying them effectively requires determining rule applicability and precedence and balancing trade-offs when rules compete. By understanding the underlying psychology, designers and evaluators enhance their ability to apply design rules. This two-part (160-minute) course explains that psychology.2018JJJeff A JohnsonUniversity of San FranciscoKnowledge Worker Tools & WorkflowsUser Research Methods (Interviews, Surveys, Observation)CHI
Making Open Data Work for Civic AdvocacyThe value of data in supporting citizen participation in processes of place-making and community building is widely recognised. While the open data movement now permits citizens to acquire governmental data relating to their communities, little to no effort is made to ensure that these datasets are accessible and interpretable by non-professionals. Through a series of community engagements spanning an 18-month period, we co-designed Data:In Place, an open source web tool which supports citizens in accessing, interpreting and making sense of open data. Leveraging visual map-based querying, citizens can access official statistics about their community, interrogate the data, and map their own data sources to create data visualisations. Reflecting on the participatory design process and the designed technology, we provide a framing to make open data work for civic advocacy.2018APAare Puussaar et al.Civic engagement and HackingCSCW
Self-Tracking for Fertility Care: Collaborative Support for a Highly Personalized ProblemInfertility is a global health concern that affects countless couples trying to conceive a child. Effective fertility treatment requires continuous monitoring of a wide range of health indicators through self-tracking. The process of collecting and interpreting data and information about fertility is complex, and much of the burden falls on women. In this study, we analyzed patient-generated content in a popular online health community dedicated to fertility issues. The objective was to understand the process in which women engage in tracking relevant information, and the challenges they face. Leveraging the Personal Informatics Model, we describe women’s self-tracking experiences during their fertility cycles. We discuss how a complex and highly personalized context leads to responsibility, pressure, and emotional burden on women performing self-tracking activities, as well as the role of collaboration in creating individualized solutions. Finally, we provide implications for technologies aiming to support women with fertility care needs.2018MFFlavio Figueiredo et al.Health and Self-TrackingCSCW
Brain-Computer Interfaces for Artistic ExpressionArtists have been using BCIs for artistic expression since the 1960s. Their interest and creativity is now increasing because of the availability of affordable BCI devices and software that does not require them to invest extensive time in getting the BCI to work or tuning it to their application. Designers of artistic BCIs are often ahead of more traditional BCI researchers in ideas on using BCIs in multimodal and multiparty contexts, where multiple users are involved, and where robustness and efficiency are not the main matters of concern. The aim of this workshop is to look at current (research) activities in BCIs for artistic expression and to identify research areas that are of interest for both BCI and HCI researchers as well as artists/designers of BCI applications.2018ANAnton Nijholt et al.University of TwenteBrain-Computer Interface (BCI) & NeurofeedbackDigital Art Installations & Interactive PerformanceCHI
Using Animation to Alleviate Overdraw in Multiclass Scatterplot MatricesThe scatterplot matrix (SPLOM) is a commonly used technique for visualizing multiclass multivariate data. However, multiclass SPLOMs have issues with overdraw (overlapping points), and most existing techniques for alleviating overdraw focus on individual scatterplots with a single class. This paper explores whether animation using flickering points is an effective way to alleviate overdraw in these multiclass SPLOMs. In a user study with 69 participants, we found that users not only performed better at identifying dense regions using animated SPLOMs, but also found them easier to interpret and preferred them to static SPLOMs. These results open up new directions for future work on alleviating overdraw for multiclass SPLOMs, and provide insights for applying animation to alleviate overdraw in other settings.2018HCHelen Chen et al.University of San FranciscoInteractive Data VisualizationVisualization Perception & CognitionCHI
Designing with the Mind in Mind: The Psychological Basis for UI Design GuidelinesUI design rules and guidelines are not simple recipes. Applying them effectively requires determining rule applicability and precedence and balancing trade-offs when rules compete. By understanding the underlying psychology, designers and evaluators enhance their ability to apply design rules. This two-part (160-minute) course explains that psychology.2018JJJeff A JohnsonUniversity of San FranciscoUser Research Methods (Interviews, Surveys, Observation)Prototyping & User TestingCHI