All

Papers

Browse and search HCI research papers from All

Active Filters

Author: 5791

12 results

JustShape: Exploring Co-Speech Gestures for Multimodal LLM-Powered 3D Parametric Modeling

Parametric modeling is a prevailing 3D modeling approach in design, architecture, and engineering. The emergence of multimodal large language models (LLMs) brings a new opportunity to lower the entry barriers to this powerful tool. However, describing 3D geometries through natural language can be fuzzy and challengi…

Runlin Duan et al.Purdue University

AgentHands: Generating Interactive Hand Gestures for Spatially Grounded Agent Conversations in XR

Communicating spatial tasks via text or speech creates ``a mental mapping gap'' that limits an agent’s expressiveness. Inspired by co-speech gestures in face-to-face conversation, we propose \textsc{AgentHands}, an LLM-powered XR system that equips agents with hands to render responses clearer and more engaging. Guide…

Ziyi Liu et al.Purdue University

AmIWrite: Exploring Scalable One-on-One Handwriting-Based Tutoring for Mathematical Problem-Solving with an LLM-Powered AI Tutor

Real-time handwriting interactions between tutors and students —where tutors observe individual problem-solving processes, provide personalized annotations, and adapt explanations based on students' work—are fundamental to effective STEM tutoring. However, scaling such personalized handwriting-based tutoring remains c…

Ziyi Liu et al.Purdue University

agentAR: Creating Augmented Reality Applications with Tool-Augmented LLM-based Autonomous Agents

Creating Augmented Reality (AR) applications requires expertise in both design and implementation, posing significant barriers to entry for non-expert users. While existing methods reduce some of this burden, they often fall short in flexibility or usability for complex or varied use cases. To address this, we introdu…

Chenfei Zhu et al.Purdue University

AdRecommended

Learn AI Coding at CodeNow

Structured lessons, hands-on projects, and continuous updates for people bringing AI into real development work.

Explore Now

GesPrompt: Leveraging Co-Speech Gestures to Augment LLM-Based Interaction in Virtual Reality

Large Language Model (LLM)-based copilots have shown great potential in Extended Reality (XR) applications. However, the user faces challenges when describing the 3D environments to the copilots due to the complexity of conveying spatial-temporal information through text or speech alone. To address this, we introduce…

Xiyun Hu et al.Purdue University

avaTTAR: Table Tennis Stroke Training with On-body and Detached Visualization in Augmented Reality

Table tennis stroke training is a critical aspect of player development. We designed a new augmented reality (AR) system, avaTTAR, for table tennis stroke training. The system provides both “on-body” (first-person view) and “detached” (third-person view) visual cues, enabling users to visualize target strokes and cor…

Dizhi Ma et al.Purdue University

ClassMeta: Designing Interactive Virtual Classmate to Promote VR Classroom Participation

Peer influence plays a crucial role in promoting classroom participation, where behaviors from active students can contribute to a collective classroom learning experience. However, the presence of these active students depends on several conditions and is not consistently available across all circumstances. Recentl…

Ziyi Liu et al.Purdue University

LearnIoTVR: An End-to-end Virtual Reality Environment Providing Authentic Learning Experiences for Internet of Things

The rapid growth of Internet-of-Things (IoT) applications has generated interest from many industries and a need for graduates with relevant knowledge. An IoT system is comprised of spatially distributed interactions between humans and various interconnected IoT components. These interactions are contextualized withi…

Zhengzhe Zhu et al.Purdue University

InstruMentAR: Auto-Generation of Augmented Reality Tutorials for Operating Digital Instruments Through Recording Embodied Demonstration

Augmented Reality tutorials, which provide necessary context by directly superimposing visual guidance on the physical referent, represent an effective way of scaffolding complex instrument operations. However, current AR tutorial authoring processes are not seamless as they require users to continuously alternate bet…

Ziyi Liu et al.Purdue University

MechARspace: An Authoring System Enabling Bidirectional Binding of AR with Toys in Real-time

Augmented Reality (AR), which blends physical and virtual worlds, presents the possibility of enhancing traditional toy design. By leveraging bidirectional virtual-physical interactions between humans and the designed artifact, such AR-enhanced toys can provide more playful and interactive experiences for traditional…

Zhengzhe Zhu et al.Purdue University

RobotAR: An Augmented Reality Compatible Teleconsulting Robotics Toolkit for Augmented Makerspaces Experiences

Distance learning is facing a critical moment finding a balance between high quality education for remote students and engaging them in hands-on learning. This is particularly relevant for project-based classrooms and makerspaces, which typically require extensive trouble-shooting and example demonstrations from instr…

Ana M Villanueva et al.Purdue University

Meta-AR-App: An Authoring Platform for Collaborative Augmented Reality in STEM Classrooms

Augmented Reality (AR) has become a valuable tool for education and training processes. Meanwhile, cloud-based technologies can foster collaboration and other interaction modalities to enhance learning. We combine the cloud capabilities with AR technologies to present Meta-AR-App, an authoring platform for collaborati…

Ana Villanueva et al.Purdue University

Paper Title	Authors	Research Topics	Paper Database	Year
emoji_events JustShape: Exploring Co-Speech Gestures for Multimodal LLM-Powered 3D Parametric Modeling Parametric modeling is a prevailing 3D modeling approach in design, architecture, and engineering. The emergence of multimodal large language models (LLMs) brings a new opportunity to lower the entry barriers to this powerful tool. However, describing 3D geometries through natural language can be fuzzy and challengi…	RD Runlin Duan et al.Purdue University
AgentHands: Generating Interactive Hand Gestures for Spatially Grounded Agent Conversations in XR Communicating spatial tasks via text or speech creates ``a mental mapping gap'' that limits an agent’s expressiveness. Inspired by co-speech gestures in face-to-face conversation, we propose \textsc{AgentHands}, an LLM-powered XR system that equips agents with hands to render responses clearer and more engaging. Guide…	ZL Ziyi Liu et al.Purdue University
AmIWrite: Exploring Scalable One-on-One Handwriting-Based Tutoring for Mathematical Problem-Solving with an LLM-Powered AI Tutor Real-time handwriting interactions between tutors and students —where tutors observe individual problem-solving processes, provide personalized annotations, and adapt explanations based on students' work—are fundamental to effective STEM tutoring. However, scaling such personalized handwriting-based tutoring remains c…	ZL Ziyi Liu et al.Purdue University
agentAR: Creating Augmented Reality Applications with Tool-Augmented LLM-based Autonomous Agents Creating Augmented Reality (AR) applications requires expertise in both design and implementation, posing significant barriers to entry for non-expert users. While existing methods reduce some of this burden, they often fall short in flexibility or usability for complex or varied use cases. To address this, we introdu…	CZ Chenfei Zhu et al.Purdue University
AdRecommended Learn AI Coding at CodeNow Structured lessons, hands-on projects, and continuous updates for people bringing AI into real development work. Explore Now
GesPrompt: Leveraging Co-Speech Gestures to Augment LLM-Based Interaction in Virtual Reality Large Language Model (LLM)-based copilots have shown great potential in Extended Reality (XR) applications. However, the user faces challenges when describing the 3D environments to the copilots due to the complexity of conveying spatial-temporal information through text or speech alone. To address this, we introduce…	XH Xiyun Hu et al.Purdue University
avaTTAR: Table Tennis Stroke Training with On-body and Detached Visualization in Augmented Reality Table tennis stroke training is a critical aspect of player development. We designed a new augmented reality (AR) system, avaTTAR, for table tennis stroke training. The system provides both “on-body” (first-person view) and “detached” (third-person view) visual cues, enabling users to visualize target strokes and cor…	DM Dizhi Ma et al.Purdue University
emoji_events ClassMeta: Designing Interactive Virtual Classmate to Promote VR Classroom Participation Peer influence plays a crucial role in promoting classroom participation, where behaviors from active students can contribute to a collective classroom learning experience. However, the presence of these active students depends on several conditions and is not consistently available across all circumstances. Recentl…	ZL Ziyi Liu et al.Purdue University
emoji_events LearnIoTVR: An End-to-end Virtual Reality Environment Providing Authentic Learning Experiences for Internet of Things The rapid growth of Internet-of-Things (IoT) applications has generated interest from many industries and a need for graduates with relevant knowledge. An IoT system is comprised of spatially distributed interactions between humans and various interconnected IoT components. These interactions are contextualized withi…	ZZ Zhengzhe Zhu et al.Purdue University
InstruMentAR: Auto-Generation of Augmented Reality Tutorials for Operating Digital Instruments Through Recording Embodied Demonstration Augmented Reality tutorials, which provide necessary context by directly superimposing visual guidance on the physical referent, represent an effective way of scaffolding complex instrument operations. However, current AR tutorial authoring processes are not seamless as they require users to continuously alternate bet…	ZL Ziyi Liu et al.Purdue University
MechARspace: An Authoring System Enabling Bidirectional Binding of AR with Toys in Real-time Augmented Reality (AR), which blends physical and virtual worlds, presents the possibility of enhancing traditional toy design. By leveraging bidirectional virtual-physical interactions between humans and the designed artifact, such AR-enhanced toys can provide more playful and interactive experiences for traditional…	ZZ Zhengzhe Zhu et al.Purdue University
RobotAR: An Augmented Reality Compatible Teleconsulting Robotics Toolkit for Augmented Makerspaces Experiences Distance learning is facing a critical moment finding a balance between high quality education for remote students and engaging them in hands-on learning. This is particularly relevant for project-based classrooms and makerspaces, which typically require extensive trouble-shooting and example demonstrations from instr…	AV Ana M Villanueva et al.Purdue University
Meta-AR-App: An Authoring Platform for Collaborative Augmented Reality in STEM Classrooms Augmented Reality (AR) has become a valuable tool for education and training processes. Meanwhile, cloud-based technologies can foster collaboration and other interaction modalities to enhance learning. We combine the cloud capabilities with AR technologies to present Meta-AR-App, an authoring platform for collaborati…	AV Ana Villanueva et al.Purdue University