GeoVisA11y: An AI-based Geovisualization Question-Answering System for Screen-Reader UsersGeovisualizations are powerful tools for communicating spatial information, but are inaccessible to screen-reader users. To address this limitation, we present GeoVisA11y, an LLM-based question-answering system that makes geovisualizations accessible through natural language interaction. The system supports map reading, analysis, interpretation and navigation by handling analytical, geospatial, visual and contextual queries. Through user studies with six screen-reader users and six sighted participants, we demonstrate that GeoVisA11y effectively bridges accessibility gaps while revealing distinct interaction patterns between user groups. We contribute: (1) an open-source, accessible geovisualization system, (2) empirical findings on query and navigation differences, and (3) a dataset of geospatial queries to inform future research on accessible data visualization.2026CLChu Li et al.University of WashingtonVisual Impairment Technologies (Screen Readers, Tactile Graphics, Braille)Interactive Data VisualizationGeospatial & Map VisualizationCHI
SPECTRA: Personalizable Sound Recognition for Deaf and Hard of Hearing Users through Interactive Machine LearningWe introduce SPECTRA, a novel pipeline for personalizable sound recognition designed to understand DHH users' needs when collecting audio data, creating a training dataset, and reasoning about the quality of a model. To evaluate the prototype, we recruited 12 DHH participants who trained personalized models for their homes. We investigated waveforms, spectrograms, interactive clustering, and data annotating to support DHH users throughout this workflow, and we explored the impact of a hands-on training session on their experience and attitudes toward sound recognition tools. Our findings reveal the potential for clustering visualizations and waveforms to enrich users' understanding of audio data and refinement of training datasets, along with data annotations to promote varied data collection. We provide insights into DHH users' experiences and perspectives on personalizing a sound recognition pipeline. Finally, we share design considerations for future interactive systems to support this population.2025SGSteven M. Goodman et al.University of Washington, Human Centered Design and EngineeringElectrical Muscle Stimulation (EMS)Visual Impairment Technologies (Screen Readers, Tactile Graphics, Braille)Deaf & Hard-of-Hearing Support (Captions, Sign Language, Vibration)CHI
NightLight: Passively Mapping Nighttime Sidewalk Light Data for Improved Pedestrian RoutingNighttime sidewalk illumination has a significant and unequal influence on where and whether pedestrians walk at night. Despite the importance of pedestrian lighting, there is currently no approach for measuring and communicating how humans experience nighttime sidewalk light levels at scale. We introduce NightLight, a new sensing approach that leverages the ubiquity of smartphones by re-appropriating the built-in light sensor ---traditionally used to adapt screen brightness---to sense pedestrian nighttime lighting conditions. We validated our technique through in-lab and street-based evaluations characterizing performance across phone orientation, phone model, and varying light levels demonstrating the ability to aggregate and map pedestrian-oriented light levels with unaltered smartphones. Additionally, to examine the impact of light level data on pedestrian route choice, we conducted a qualitative user study with 13 participants using a standard map vs. one with pedestrian lighting data from NightLight Our findings demonstrate that people changed their routes in preference of well-light routes during nighttime walking. Our work has implications for expanding personalized navigation and pedestrian route choice and passive urban sensing.2025JBJoseph Breda et al.University of Washington, Paul G. Allen School of Computer Science & EngineeringContext-Aware ComputingSmart Cities & Urban SensingCHI
Accessibility for Whom? Perceptions of Mobility Barriers Across Disability Groups and Implications for Designing Personalized MapsToday’s mapping tools fail to address the varied experiences of different mobility device users. This paper presents a large-scale online survey exploring how five mobility groups—users of canes, walkers, mobility scooters, manual wheelchairs, and motorized wheelchairs—perceive sidewalk barriers and differences therein. Using 52 sidewalk barrier images, respondents evaluated their confidence in navigating each scenario. Our findings (N=190) reveal variations in barrier perceptions across groups, while also identifying shared concerns. To further demonstrate the value of this data, we showcase its use in two custom prototypes: a visual analytics tool and a personalized routing tool. Our survey findings and open dataset advance work in accessibility-focused maps, routing algorithms, and urban planning.2025CLChu Li et al.University of Washington, Allen School of Computer ScienceUniversal & Inclusive DesignGeospatial & Map VisualizationPedestrian & Cyclist SafetyCHI
“I never realized sidewalks were a big deal”: A Case Study of a Community-Driven Sidewalk Accessibility Assessment using Project SidewalkDespite decades of effort, pedestrian infrastructure in cities continues to be unsafe or inaccessible to people with disabilities. In this paper, we examine the potential of community-driven digital civics to assess sidewalk accessibility through a deployment study of an open-source crowdsourcing tool called Project Sidewalk. We explore Project Sidewalk's potential as a platform for civic learning and service. Specifically, we assess its effectiveness as a tool for community members to learn about human mobility, urban planning, and accessibility advocacy. Our findings demonstrate that community-driven digital civics can support accessibility advocacy and education, raise community awareness, and drive pro-social behavioral change. We also outline key considerations for deploying digital civic tools in future community-led accessibility initiatives.2024CLChu Li et al.University of WashingtonUniversal & Inclusive DesignPedestrian & Cyclist SafetyCommunity Engagement & Civic TechnologyCHI
"Caption it in an Accessible Way That is Also Enjoyable": Characterizing User-Driven Captioning Practices on TikTokAs user-generated video dominates media landscapes, it poses an accessibility challenge. While disability advocacy groups globally have secured hard-won accessibility regulations for broadcast media, no such regulation of user-generated content exists. Yet, one major player in this shift, TikTok, has a culture of user-generated, creative captioning. We sought to understand how TikTok videos are captioned and the impact current practices have on those who need captions to access audio content. Therefore, we conducted a content analysis of 300 open-captioned TikToks and contextualized these findings by interviewing nine caption users. We found that the current state of TikTok captioning does facilitate access to the platform but that a user-generated, social video-specific standard for captioning could improve caption quality and expand access. We contribute an empirical account of the state of TikTok captioning and outline steps toward a standard for user-generated captioning.2024EMEmma J McDonnell et al.University of WashingtonDeaf & Hard-of-Hearing Support (Captions, Sign Language, Vibration)Game AccessibilityCHI
GazePointAR: A Context-Aware Multimodal Voice Assistant for Pronoun Disambiguation in Wearable Augmented RealityVoice assistants (VAs) like Siri and Alexa are transforming human-computer interaction; however, they lack awareness of users' spatiotemporal context, resulting in limited performance and unnatural dialogue. We introduce GazePointAR, a fully-functional context-aware VA for wearable augmented reality that leverages eye gaze, pointing gestures, and conversation history to disambiguate speech queries. With GazePointAR, users can ask "what's over there?" or "how do I solve this math problem?" simply by looking and/or pointing. We evaluated GazePointAR in a three-part lab study (N=12): (1) comparing GazePointAR to two commercial systems, (2) examining GazePointAR's pronoun disambiguation across three tasks; (3) and an open-ended phase where participants could suggest and try their own context-sensitive queries. Participants appreciated the naturalness and human-like nature of pronoun-driven queries, although sometimes pronoun use was counter-intuitive. We then iterated on GazePointAR and conducted a first-person diary study examining how GazePointAR performs in-the-wild. We conclude by enumerating limitations and design considerations for future context-aware VAs.2024JLJaewook Lee et al.University of WashingtonEye Tracking & Gaze InteractionVoice User Interface (VUI) DesignAR Navigation & Context AwarenessCHI
Playing on Hard Mode: Accessibility, Difficulty and Joy in Video Game Adoption for Gamers with DisabilitiesVideo games often pose accessibility barriers to gamers with disabilities, yet there is no standard method for identifying which games have barriers, what those barriers are, and whether and how they can be overcome. We propose and explore three phases of the “game adoption process”: Discovery, Evaluation, and Adaptation. To advance understanding of how gamers with disabilities experience this process, the resources and strategies they use, and the challenges experienced, we conducted an interview study with thirteen gamers with disabilities with differing backgrounds. We then engage with existing theories of consequence-based accessibility, of difficulty, and of identity-based gaming to better understand how these processes manifest “access difficulty” and to characterize the experience of “disabled gaming.” Finally, we present design recommendations for game developers and distributors to better support gamers with disabilities in the game adoption process by engaging with community-made resources, supporting socially-created access, and creating customizable experiences with opportunities for unconventional play.2024JMJesse J Martinez et al.University of WashingtonAccessible GamingGame UX & Player BehaviorGame AccessibilityCHI
“Easier or Harder, Depending on Who the Hearing Person Is”: Codesigning Videoconferencing Tools for Small Groups with Mixed Hearing StatusWith improvements in automated speech recognition and increased use of videoconferencing, real-time captioning has changed significantly. This shift toward broadly available but less accurate captioning invites exploration of the role hearing conversation partners play in shaping the accessibility of a conversation to d/Deaf and hard of hearing (DHH) captioning users. While recent work has explored DHH individuals’ videoconferencing experiences with captioning, we focus on established groups’ current practices and priorities for future tools to support more accessible online conversations. Our study consists of three codesign sessions, conducted with four groups (17 participants total, 10 DHH, 7 hearing). We found that established groups crafted social accessibility norms that met their relational contexts. We also identify promising directions for future captioning design, including the need to standardize speaker identification and customization, opportunities to provide behavioral feedback during a conversation, and ways that videoconferencing platforms could enable groups to set and share norms.2023EMEmma J McDonnell et al.University of WashingtonConversational ChatbotsDeaf & Hard-of-Hearing Support (Captions, Sign Language, Vibration)Universal & Inclusive DesignCHI
Visualizing Urban Accessibility: Investigating Multi-Stakeholder Perspectives through a Map-based Design Probe StudyUrban accessibility assessments are challenging: they involve varied stakeholders across decision-making contexts while serving a diverse population of people with disabilities. To better support urban accessibility assessment using data visualizations, we conducted a three-part interview study with 25 participants across five stakeholder groups using map visualization probes. We present a multi-stakeholder analysis of visualization needs and sensemaking processes to explore how interactive visualizations can support stakeholder decision making. In particular, we elaborate how stakeholders’ varying levels of familiarity with accessibility, geospatial analysis, and specific geographic locations influences their sensemaking needs. We then contribute 10 design considerations for geovisual analytic tools for urban accessibility communication, planning, policymaking, and advocacy.2022MSManaswi Saha et al.University of WashingtonGeospatial & Map VisualizationSmart Cities & Urban SensingInclusive DesignCHI
Pedagogical Strategies for Reflection in Project-based HCI Education with End UsersAs HCI pedagogy research grows, so too does an emerging set of evidence-based teaching and curricular recommendations. Yet, few studies have implemented and examined these recommendations in the classroom. In this paper, we present a Research Through Design investigation of a studio-based HCI course, which was revised based on HCI education literature. Drawing on reflection surveys, video recordings of student-led user sessions, final project artifacts, and student interviews, we explore how students responded to key educational changes, the strategies that supported and hindered their reflective practices, and how reflection afforded new student insights. Our findings highlight the utility of video-based reflection exercises to support student learning in designing and running user sessions, the importance of multi-faceted reflection prompts, and how students noticed moments of inclusion and exclusion by attending to users’ non-verbal cues. Additionally, we empirically demonstrate the importance of implementing and studying HCI education research recommendations in the classroom.2021WRWendy Roldan et al.Collaborative Learning & Peer TeachingUser Research Methods (Interviews, Surveys, Observation)Prototyping & User TestingDIS
Opportunities and Challenges in Involving Users in Project-Based HCI EducationUsers are fundamental to HCI. However, little is known about how HCI education introduces students to working with users, particularly those different from themselves. To better understand design students' engagement, reactions, and reflections with users, we investigate a case study of a graduate-level 10-week prototyping studio course that partnered with a children's co-design team. HCI students participated in two co-design sessions with children to design a STEM learning experience for youth. We conducted participant observations, interviews with 14 students, and analyzed final artifacts. Our findings demonstrate the communication challenges and strategies students experienced, how students observed issues of power dynamics, and students' perceived value in engaging with users. We contribute empirical evidence of how HCI students directly interact with target users, principles for reflective HCI pedagogy, and highlight the need for more intentional investigation into HCI educational practice.2020WRWendy Roldan et al.University of WashingtonSTEM Education & Science CommunicationParticipatory DesignPrototyping & User TestingCHI
ARMath: Augmenting Everyday Life with Math LearningWe introduce ARMath, a mobile Augmented Reality (AR) system that allows ch ildren to discover mathematical concepts in familiar, ord inary objects and engage with math problems in meaningful contexts. Leveraging advanced computer vision, ARMath recognizes everyday objects, visualizes their mathematical attributes, and turns them into tangible or virtual manipulatives. Using the manipulatives, children can solve problems that situate math operations or concepts in specific everyday contexts. Informed by four participatory design sessions with teachers and children, we developed five ARMath modules to support basic arithmetic and 2D geometry. We also conducted an exploratory evaluation of ARMath with 27 children (ages 5-8) at a local children's museum. Our findings demonstrate how ARMath engages children in math learning, how failures in AI can be used as learning opportunities, and challenges that children face when using ARMath.2020SKSeokbin kang et al.University of MarylandAR Navigation & Context AwarenessK-12 Digital Education ToolsSTEM Education & Science CommunicationCHI
Evaluating Smartwatch-based Sound Feedback for Deaf and Hard-of-hearing Users Across ContextsWe present a qualitative study with 16 deaf and hard of hearing (DHH) participants examining reactions to smartwatch-based visual + haptic sound feedback designs. In Part 1, we conducted a Wizard-of-Oz (WoZ) evaluation of three smartwatch feedback techniques (visual alone, visual + simple vibration, and visual + tacton) and investigated vibrational patterns (tactons) to portray sound loudness, direction, and identity. In Part 2, we visited three public or semi-public locations where we demonstrated sound feedback on the smartwatch in situ to examine contextual influences and explore sound filtering options. Our findings characterize uses for vibration in multimodal sound awareness, both for push notification and for immediately actionable sound information displayed through vibrational patterns (tactons). In situ experiences caused participants to request sound filtering — particularly to limit haptic feedback — as a method for managing soundscape complexity. Additional concerns arose related to learnability, possibility of distraction, and system trust. Our findings have implications for future portable sound awareness systems.2020SGSteven Goodman et al.University of WashingtonDeaf & Hard-of-Hearing Support (Captions, Sign Language, Vibration)Smartwatches & Fitness BandsCHI