Xiyun Hu

schoolPurdue Universitybar_chart#577 Global Rank

Total Papers

HCI Rank

#577

Research Areas

Years Active

Recent Publications

JustShape: Exploring Co-Speech Gestures for Multimodal LLM-Powered 3D Parametric Modeling

Parametric modeling is a prevailing 3D modeling approach in design, architecture, and engineering. The emergence of multimodal large language models (LLMs) brings a new opportunity to lower the entry barriers to this powerful tool. However, describing 3D geometries through natural language can be fuzzy and challengi…

CHI '261st AuthorAR Navigation & Context Awareness +2

ARify: Leveraging Narrated Instructional Videos to Create Augmented Reality Tutorials for Procedural Tasks

Augmented Reality (AR) tutorials enhance procedural task learning by providing situated, step-by-step guidance. Yet, creating such tutorials requires AR authoring expertise, posing a significant entry barrier. To lower this barrier, we introduce ARify, an authoring system that semi-automatically transforms narrated in…

CHI '264th AuthorHuman Pose & Activity Recognition +2

AgentCoach: LLM-Based Adaptive Coaching Feedback for Motor Skill Learning

We present AgentCoach, an LLM-powered system that provides adaptive feedback for motor skill learning from tutorial videos. The system works by extracting key coaching points (CPs) and compiling CP-specific evaluators that map each cue to measurable kinematic parameters. This process allows AgentCoach to connect high-…

CHI '265th+ AuthorHand Gesture Recognition +2

AmIWrite: Exploring Scalable One-on-One Handwriting-Based Tutoring for Mathematical Problem-Solving with an LLM-Powered AI Tutor

Real-time handwriting interactions between tutors and students —where tutors observe individual problem-solving processes, provide personalized annotations, and adapt explanations based on students' work—are fundamental to effective STEM tutoring. However, scaling such personalized handwriting-based tutoring remains c…

UIST '253rd AuthorAR Navigation & Context Awareness +2

agentAR: Creating Augmented Reality Applications with Tool-Augmented LLM-based Autonomous Agents

Creating Augmented Reality (AR) applications requires expertise in both design and implementation, posing significant barriers to entry for non-expert users. While existing methods reduce some of this burden, they often fall short in flexibility or usability for complex or varied use cases. To address this, we introdu…

DIS '251st AuthorHand Gesture Recognition +2

GesPrompt: Leveraging Co-Speech Gestures to Augment LLM-Based Interaction in Virtual Reality

Large Language Model (LLM)-based copilots have shown great potential in Extended Reality (XR) applications. However, the user faces challenges when describing the 3D environments to the copilots due to the complexity of conveying spatial-temporal information through text or speech alone. To address this, we introduce…

UIST '242nd AuthorFull-Body Interaction & Embodied Input +2

avaTTAR: Table Tennis Stroke Training with On-body and Detached Visualization in Augmented Reality

Table tennis stroke training is a critical aspect of player development. We designed a new augmented reality (AR) system, avaTTAR, for table tennis stroke training. The system provides both “on-body” (first-person view) and “detached” (third-person view) visual cues, enabling users to visualize target strokes and cor…

CHI '245th+ AuthorHonorable MentionSocial & Collaborative VR +2

ClassMeta: Designing Interactive Virtual Classmate to Promote VR Classroom Participation

Peer influence plays a crucial role in promoting classroom participation, where behaviors from active students can contribute to a collective classroom learning experience. However, the presence of these active students depends on several conditions and is not consistently available across all circumstances. Recentl…

CHI '232nd AuthorShape-Changing Interfaces & Soft Robotic Materials +1

Ubi Edge: Authoring Edge-Based Opportunistic Tangible User Interfaces in Augmented Reality

Edges are one of the most ubiquitous geometric features of physical objects. They provide accurate haptic feedback and easy-to-track features for camera systems, making them an ideal basis for Tangible User Interfaces (TUI) in Augmented Reality (AR). We introduce Ubi Edge, an AR authoring tool that allows end-users to…

UIST '223rd AuthorHand Gesture Recognition +3

ARnnotate: An Augmented Reality Interface for Collecting Custom Dataset of 3D Hand-Object Interaction Pose Estimation

Vision-based 3D pose estimation has substantial potential in hand-object interaction applications and requires user-specified datasets to achieve robust performance. We propose ARnnotate, an Augmented Reality (AR) interface enabling end-users to create custom data using a hand-tracking-capable AR device. Unlike other…

View all 13 papers →

Quick Actions

Share this page

ios_share

X in mail

https://hci.top/en/authors/8930?source=all