All

Papers

Browse and search HCI research papers from All

Active Filters

Author: 13179

11 results

AgentHands: Generating Interactive Hand Gestures for Spatially Grounded Agent Conversations in XR

Communicating spatial tasks via text or speech creates ``a mental mapping gap'' that limits an agent’s expressiveness. Inspired by co-speech gestures in face-to-face conversation, we propose \textsc{AgentHands}, an LLM-powered XR system that equips agents with hands to render responses clearer and more engaging. Guide…

Ziyi Liu et al.Purdue University

Sensible Agent: A Framework for Unobtrusive Interaction with Proactive AR Agents

Proactive AR agents promise context-aware assistance, but their interactions often rely on explicit voice prompts or responses, which can be disruptive or socially awkward. We introduce Sensible Agent,a framework designed for unobtrusive interaction with these proactive agents. Sensible Agent dynamically adapts both “…

Geonsun Lee et al.University Of Maryland

Thing2Reality: Enabling Spontaneous Creation of 3D Objects from 2D Content using Generative AI in XR Meetings

During remote communication, participants often share both digital and physical content, such as product designs, digital assets, and environments, to enhance mutual understanding. Recent advances in augmented communication have facilitated users to swiftly create and share digital 2D copies of physical objects from v…

Erzhen Hu et al.University of Virginia

DialogLab: Authoring, Simulating, and Testing Dynamic Human-AI Group Conversations

Designing compelling multi-party conversations involving both humans and AI agents presents significant challenges, particularly in balancing scripted structure with emergent, human-like interactions. We introduce DialogLab, a prototyping toolkit for authoring, simulating, and testing hybrid human-AI dialogues. Dialog…

Erzhen Hu et al.University of Virginia

AdRecommended

Learn AI Coding at CodeNow

Structured lessons, hands-on projects, and continuous updates for people bringing AI into real development work.

Explore Now

InstructPipe: Generating Visual Blocks Pipelines with Human Instructions and LLMs

Visual programming has the potential of providing novice programmers with a low-code experience to build customized processing pipelines. Existing systems typically require users to build pipelines from scratch, implying that novice users are expected to set up and link appropriate nodes from a blank workspace. In thi…

Zhongyi Zhou et al.Google

Augmented Object Intelligence with XR-Objects

Seamless integration of physical objects as interactive digital entities remains a challenge for spatial computing. This paper explores Augmented Object Intelligence (AOI) in the context of XR, an interaction paradigm that aims to blur the lines between digital and physical by equipping real-world objects with the abi…

Mustafa Doga Dogan et al.Adobe

ChatDirector: Enhancing Video Conferencing with Space-Aware Scene Rendering and Speech-Driven Layout Transition

Remote video conferencing systems (RVCS) are widely adopted in personal and professional communication. However, they often lack the co-presence experience of in-person meetings. This is largely due to the absence of intuitive visual cues and clear spatial relationships among remote participants, which can lead to spe…

Xun Qian et al.Purdue University

Human I/O: Towards a Unified Approach to Detecting Situational Impairments

Situationally Induced Impairments and Disabilities (SIIDs) can significantly hinder user experience in contexts such as poor lighting, noise, and multi-tasking. While prior research has introduced algorithms and systems to address these impairments, they predominantly cater to specific tasks or environments and fail t…

Xingyu Bruce Liu et al.University of California

UI Mobility Control in XR: Switching UI Positionings between Static, Dynamic, and Self Entities

Extended reality (XR) has the potential for seamless user interface (UI) transitions across people, objects, and environments. However, the design space, applications, and common practices of 3D UI transitions remain underexplored. To address this gap, we conducted a need-finding study with 11 participants, identifyin…

Siyou Pei et al.University of California

RetroSphere: Self-Contained Passive 3D Controller Tracking for Augmented Reality

Advanced AR/VR headsets often have a dedicated depth sensor or multiple cameras, high processing power, and a high-capacity battery to track hands or controllers. However, these approaches are not compatible with the small form factor and limited thermal capacity of lightweight AR devices. In this paper, we present Re…

Ananta Narayanan Balaji et al.National University of Singapore

DepthLab: Real-time 3D Interaction with Depth Maps for Mobile Augmented Reality

Mobile devices with passive depth sensing capabilities are ubiquitous, and recently active depth sensors have become available on some tablets and AR/VR devices. Although realtime depth data is accessible, its rich value to mainstream AR applications has been sorely under-explored. Adoption of depth-based UX has been…

Ruofei Du et al.Google

Paper Title	Authors	Research Topics	Paper Database	Year
AgentHands: Generating Interactive Hand Gestures for Spatially Grounded Agent Conversations in XR Communicating spatial tasks via text or speech creates ``a mental mapping gap'' that limits an agent’s expressiveness. Inspired by co-speech gestures in face-to-face conversation, we propose \textsc{AgentHands}, an LLM-powered XR system that equips agents with hands to render responses clearer and more engaging. Guide…	ZL Ziyi Liu et al.Purdue University
Sensible Agent: A Framework for Unobtrusive Interaction with Proactive AR Agents Proactive AR agents promise context-aware assistance, but their interactions often rely on explicit voice prompts or responses, which can be disruptive or socially awkward. We introduce Sensible Agent,a framework designed for unobtrusive interaction with these proactive agents. Sensible Agent dynamically adapts both “…	GL Geonsun Lee et al.University Of Maryland
Thing2Reality: Enabling Spontaneous Creation of 3D Objects from 2D Content using Generative AI in XR Meetings During remote communication, participants often share both digital and physical content, such as product designs, digital assets, and environments, to enhance mutual understanding. Recent advances in augmented communication have facilitated users to swiftly create and share digital 2D copies of physical objects from v…	EH Erzhen Hu et al.University of Virginia
DialogLab: Authoring, Simulating, and Testing Dynamic Human-AI Group Conversations Designing compelling multi-party conversations involving both humans and AI agents presents significant challenges, particularly in balancing scripted structure with emergent, human-like interactions. We introduce DialogLab, a prototyping toolkit for authoring, simulating, and testing hybrid human-AI dialogues. Dialog…	EH Erzhen Hu et al.University of Virginia
AdRecommended Learn AI Coding at CodeNow Structured lessons, hands-on projects, and continuous updates for people bringing AI into real development work. Explore Now
emoji_events InstructPipe: Generating Visual Blocks Pipelines with Human Instructions and LLMs Visual programming has the potential of providing novice programmers with a low-code experience to build customized processing pipelines. Existing systems typically require users to build pipelines from scratch, implying that novice users are expected to set up and link appropriate nodes from a blank workspace. In thi…	ZZ Zhongyi Zhou et al.Google
Augmented Object Intelligence with XR-Objects Seamless integration of physical objects as interactive digital entities remains a challenge for spatial computing. This paper explores Augmented Object Intelligence (AOI) in the context of XR, an interaction paradigm that aims to blur the lines between digital and physical by equipping real-world objects with the abi…	MD Mustafa Doga Dogan et al.Adobe
ChatDirector: Enhancing Video Conferencing with Space-Aware Scene Rendering and Speech-Driven Layout Transition Remote video conferencing systems (RVCS) are widely adopted in personal and professional communication. However, they often lack the co-presence experience of in-person meetings. This is largely due to the absence of intuitive visual cues and clear spatial relationships among remote participants, which can lead to spe…	XQ Xun Qian et al.Purdue University
emoji_events Human I/O: Towards a Unified Approach to Detecting Situational Impairments Situationally Induced Impairments and Disabilities (SIIDs) can significantly hinder user experience in contexts such as poor lighting, noise, and multi-tasking. While prior research has introduced algorithms and systems to address these impairments, they predominantly cater to specific tasks or environments and fail t…	XL Xingyu Bruce Liu et al.University of California
UI Mobility Control in XR: Switching UI Positionings between Static, Dynamic, and Self Entities Extended reality (XR) has the potential for seamless user interface (UI) transitions across people, objects, and environments. However, the design space, applications, and common practices of 3D UI transitions remain underexplored. To address this gap, we conducted a need-finding study with 11 participants, identifyin…	SP Siyou Pei et al.University of California
RetroSphere: Self-Contained Passive 3D Controller Tracking for Augmented Reality Advanced AR/VR headsets often have a dedicated depth sensor or multiple cameras, high processing power, and a high-capacity battery to track hands or controllers. However, these approaches are not compatible with the small form factor and limited thermal capacity of lightweight AR devices. In this paper, we present Re…	AB Ananta Narayanan Balaji et al.National University of Singapore
DepthLab: Real-time 3D Interaction with Depth Maps for Mobile Augmented Reality Mobile devices with passive depth sensing capabilities are ubiquitous, and recently active depth sensors have become available on some tablets and AR/VR devices. Although realtime depth data is accessible, its rich value to mainstream AR applications has been sorely under-explored. Adoption of depth-based UX has been…	RD Ruofei Du et al.Google