Capability at a Glance: Design Guidelines for Intuitive Avatars Communicating Augmented Actions in Virtual RealityVirtual Reality (VR) enables users to engage with capabilities beyond human limitations, but it is not always obvious how to trigger these capabilities. Taking the lens of Affordance, we believe avatar design is the key to solving this issue, which ideally should communicate its capabilities and how to activate them.…2026YLYang K. Lu et al.Hong Kong University of Science and TechnologyIdentity & Avatars in XRSocial & Collaborative VRImmersion & Presence ResearchCHI
Branch Explorer: Leveraging Branching Narratives to Support Interactive 360° Video Viewing for Blind and Low Vision Users360° videos enable users to freely choose their viewing paths, but blind and low vision (BLV) users are often excluded from this interactive experience. To bridge this gap, we present Branch Explorer, a system that transforms 360° videos into branching narratives—stories that dynamically unfold based on viewer choices…2025SXShuchang Xu et al.Hong Kong University of Science and Technology360° Video & Panoramic ContentAccessible GamingUIST
From Following to Understanding: Investigating the Role of Reflective Prompts in AR-Guided Tasks to Promote User UnderstandingAugmented Reality (AR) is a promising medium for guiding users through tasks, yet its impact on fostering deeper task understanding remains underexplored. This paper investigates the impact of reflective prompts---strategic questions that encourage users to challenge assumptions, connect actions to outcomes, and consi…2025NZNandi Zhang et al.University of CalgaryAR Navigation & Context AwarenessPrototyping & User TestingCHI
DanmuA11y: Making Time-Synced On-Screen Video Comments (Danmu) Accessible to Blind and Low Vision Users via Multi-Viewer Audio DiscussionsBy overlaying time-synced user comments on videos, Danmu creates a co-watching experience for online viewers. However, its visual-centric design poses significant challenges for blind and low vision (BLV) viewers. Our formative study identified three primary challenges that hinder BLV viewers' engagement with Danmu: t…2025SXShuchang Xu et al.Hong Kong University of Science and TechnologyVoice AccessibilityAccessible GamingUniversal & Inclusive DesignCHI
Modeling the Impact of Visual Stimuli on Redirection Noticeability with Gaze Behavior in Virtual RealityWhile users could embody virtual avatars that mirror their physical movements in Virtual Reality, these avatars' motions can be redirected to enable novel interactions. Excessive redirection, however, could break the user's sense of embodiment due to perceptual conflicts between vision and proprioception. While prior…2025ZLZhipeng Li et al.ETH ZurichEye Tracking & Gaze InteractionMixed Reality WorkspacesImmersion & Presence ResearchCHI
Computational Trichromacy Reconstruction: Empowering the Color-Vision Deficient to Recognize Colors Using Augmented RealityWe propose an assistive technology that helps individuals with Color Vision Deficiencies (CVD) to recognize/name colors. A dichromat's color perception is a reduced two-dimensional (2D) subset of a normal trichromat's three dimensional color (3D) perception, leading to confusion when visual stimuli that appear ident…2024YZYuhao Zhu et al.University of RochesterAR Navigation & Context AwarenessVisual Impairment Technologies (Screen Readers, Tactile Graphics, Braille)UIST
Auptimize: Optimal Placement of Spatial Audio Cues for Extended RealitySpatial audio in Extended Reality (XR) provides users with better awareness of where virtual elements are placed, and efficiently guides them to events such as notifications, system alerts from different windows, or approaching avatars. Humans, however, are inaccurate in localizing sound cues, especially with multiple…2024HCHyunsung Cho et al.Carnegie Mellon UniversitySocial & Collaborative VRImmersion & Presence ResearchUIST
Memory Reviver: Supporting Photo-Collection Reminiscence for People with Visual Impairment via a Proactive ChatbotReminiscing with photo collections offers significant psychological benefits but poses challenges for people with visual impairment (PVI). Their current reliance on sighted help restricts the flexibility of this activity. In response, we explored using a chatbot in a preliminary study. We identified two primary challe…2024SXShuchang Xu et al.Hong Kong University of Science and TechnologyConversational ChatbotsCognitive Impairment & Neurodiversity (Autism, ADHD, Dyslexia)UIST
Predicting the Noticeability of Dynamic Virtual Elements in Virtual RealityWhile Virtual Reality (VR) systems can present virtual elements such as notifications anywhere, designing them so they are not missed by or distracting to users is highly challenging for content creators. To address this challenge, we introduce a novel approach to predict the noticeability of virtual elements. It comp…2024ZLZhipeng Li et al.ETH ZurichImmersion & Presence ResearchHuman-LLM CollaborationCHI
MineXR: Mining Personalized Extended Reality InterfacesExtended Reality (XR) interfaces offer engaging user experiences, but their effective design requires a nuanced understanding of user behavior and preferences. This knowledge is challenging to obtain without the widespread adoption of XR devices. We introduce MineXR, a design mining workflow and data analysis platform…2024HCHyunsung Cho et al.Carnegie Mellon UniversityMixed Reality WorkspacesImmersion & Presence ResearchInteractive Data VisualizationCHI
AdaptiveVoice: Cognitively Adaptive Voice Interface for Driving AssistanceCurrent voice assistants present messages in a predefined format without considering users’ mental states. This paper presents an optimization-based approach to alleviate this issue which adjusts the level of details and speech speed of the voice messages according to the estimated cognitive load of the user. In th…2024SWShaoyue Wen et al.New York UniversityHead-Up Display (HUD) & Advanced Driver Assistance Systems (ADAS)Voice User Interface (VUI) DesignCHI
Time2Stop: Adaptive and Explainable Human-AI Loop for Smartphone Overuse InterventionDespite a rich history of investigating smartphone overuse intervention techniques, AI-based just-in-time adaptive intervention (JITAI) methods for overuse reduction are lacking. We develop Time2Stop, an intelligent, adaptive, and explainable JITAI system that leverages machine learning to identify optimal interventio…2024AOAdiba Orzikulova et al.Korea Advanced Institute of Science and TechnologyExplainable AI (XAI)AI-Assisted Decision-Making & AutomationNotification & Interruption ManagementCHI
DRG-Keyboard: Enabling Subtle Gesture Typing on the Fingertip with Dual IMU Rings"We present DRG-Keyboard, a gesture keyboard enabled by dual IMU rings, allowing the user to swipe the thumb on the index fingertip to perform word gesture typing as if typing on a miniature QWERTY keyboard. With dual IMUs attached to the user's thumb and index finger, DRG-Keyboard can 1) measure the relative attitude…2023CLChen Liang et al.Tsinghua UniversityVibrotactile Feedback & Skin StimulationHaptic WearablesHand Gesture RecognitionUbiComp
Modeling the Trade-off of Privacy Preservation and Activity Recognition on Low-Resolution ImagesA computer vision system using low-resolution image sensors can provide intelligent services (e.g., activity recognition) but preserve unnecessary visual privacy information from the hardware level. However, preserving visual privacy and enabling accurate machine recognition have adversarial needs on image resolution.…2023YWYuntao Wang et al.Tsinghua UniversityHuman Pose & Activity RecognitionPrivacy Perception & Decision-MakingCHI
HandAvatar: Embodying Non-Humanoid Virtual Avatars through HandsWe propose HandAvatar to enable users to embody non-humanoid avatars using their hands. HandAvatar leverages the high dexterity and coordination of users' hands to control virtual avatars, enabled through our novel approach for automatically-generated joint-to-joint mappings. We contribute an observation study to unde…2023YJYu Jiang et al.Tsinghua UniversityHand Gesture RecognitionMixed Reality WorkspacesIdentity & Avatars in XRCHI
Selecting Real-World Objects via User-Perspective Phone OcclusionPerceiving the region of interest (ROI) and target object by smartphones from the user's first-person perspective can enable diverse spatial interactions. In this paper, we propose a novel ROI input method and a target selecting method for smartphones by utilizing the user-perspective phone occlusion. This concept of…2023YQYue Qin et al.Tsinghua UniversityHand Gesture RecognitionEye Tracking & Gaze InteractionContext-Aware ComputingCHI
Color-to-Depth Mappings as Depth Cues in Virtual RealityDespite significant improvements to Virtual Reality (VR) technologies, most VR displays are fixed focus and depth perception is still a key issue that limits the user experience and the interaction performance. To supplement humans' inherent depth cues (e.g., retinal blur, motion parallax), we investigate users' perce…2022ZLZhipeng Li et al.ETH ZurichImmersion & Presence ResearchMedical & Scientific Data VisualizationUIST
FaceOri: Tracking Head Position and Orientation Using Ultrasonic Ranging on EarphonesFace orientation can often indicate users’ intended interaction target. In this paper, we propose FaceOri, a novel face tracking technique based on acoustic ranging using earphones. FaceOri can leverage the speaker on a commodity device to emit an ultrasonic chirp, which is picked up by the set of microphones on the u…2022YWYuntao Wang et al.Tsinghua UniversityEye Tracking & Gaze InteractionContext-Aware ComputingCHI
Towards Understanding Diminished RealityDiminished reality (DR) refers to the concept of removing content from a user's visual environment. While its implementation is becoming feasible, it is still unclear how users perceive and interact in DR-enabled environments and what applications it benefits. To address this challenge, we first conduct a formative st…2022YCYi Fei Cheng et al.Carnegie Mellon UniversityMixed Reality WorkspacesImmersion & Presence ResearchContext-Aware ComputingCHI
ReflecTrack: Enabling 3D Acoustic Position Tracking Using Commodity Dual-Microphone Smartphones3D position tracking on smartphones has the potential to unlock a variety of novel applications, but has not been made widely available due to limitations in smartphone sensors. In this paper, we propose ReflecTrack, a novel 3D acoustic position tracking method for commodity dual-microphone smartphones. A ubiquitous s…2021YZYuzhou Zhuang et al.Tsinghua UniversityFull-Body Interaction & Embodied InputBiosensors & Physiological MonitoringUIST