Reimagining Wearable AR Gesture Design: Physical Therapy Reasoning in Everyday Contexts

Paper Title

Reimagining Wearable AR Gesture Design: Physical Therapy Reasoning in Everyday Contexts

Publication Info

• Topic area: Gesture design for lightweight augmented reality (AR) glasses with a focus on ergonomics and social legibility.
• Keywords: Augmented reality, gesture design, ergonomics, physical therapy, social legibility, wearable AR, everyday interaction, gesture substitution, sustainable gestures, AR glasses.

Background and Problem

• Problem / challenge: Existing gesture vocabularies for AR glasses are largely derived from VR or industrial AR systems, prioritizing recognizer accuracy over ergonomic sustainability and social legibility. These gestures are often fatiguing, biomechanically unsafe, and socially inappropriate for everyday use.
• Significance: As lightweight AR glasses transition into everyday environments, sustainable and socially coherent gesture vocabularies are essential to ensure usability, comfort, and acceptance in diverse contexts.
• Motivation and related work: Prior work in gesture design has focused on expert-driven, computation-driven, or user-driven methods, but these approaches often neglect long-term ergonomic safety and social visibility. The study builds on these gaps by integrating physical therapists (PTs) as movement-safety experts to address the unique challenges of everyday AR gesture design.

Solution

• Proposed approach: The study introduces a PT-informed gesture design framework, leveraging clinical reasoning to create sustainable and socially legible gestures for lightweight AR glasses.
• Novelty:
  1. Development of a PT-informed gesture translation method to systematically substitute fragile gestures with sustainable alternatives.
  2. Introduction of the Everyday-AR Golden Ergonomic Canvas, a chest-level, elbow-anchored workspace for sustainable gesture design.
  3. Creation of a stage-aware social legibility framework, linking ergonomic stability with social appropriateness across six everyday contexts.
• Procedure and key techniques:
  • Reviewed 104 AR applications to identify 15 common gesture intents.
  • Designed an on-device gesture elicitation system using Snap Spectacles.
  • Conducted a three-stage study with 10 licensed PTs to refine gestures:
    1. Intuitive execution of gestures.
    2. PT-guided ergonomic substitutions.
    3. Contextual card sorting to align gestures with social and situational demands.

Results

• Concrete findings:
  • Identified four biomechanical principles for sustainable gestures: Joint-Rotation Substitution, Shoulder-Line Workspace Constraint, Proximal–Distal Stability, and Fine–Gross Motor Integration.
  • Proposed ergonomic substitutions for 15 gesture intents, transforming intuitive but fragile gestures into sustainable forms.
  • Synthesized findings into the Everyday-AR Golden Ergonomic Canvas, defining a chest-level, elbow-anchored ±30° workspace for repeatable and fatigue-resistant gestures.
  • Developed a dramaturgical matrix of six everyday contexts (e.g., front-stage, back-stage, off-stage) to guide social legibility.
• Advantage over baselines:
  • Reframed gesture design from "naturalness" to "sustainability," addressing fatigue, joint safety, and social coherence.
  • Provided a recognizer-agnostic framework applicable across AR platforms and sensing modalities.
• Experiments / evaluation:
  • Conducted a qualitative study with 10 PTs (M = 14.8 years of experience) using Snap Spectacles.
  • Data sources included video recordings, first-person audio/video, and structured observer notes.
  • Analysis revealed consistent PT reasoning and validated the proposed ergonomic and social frameworks.
• Limitations and future work:
  • The study does not include quantitative evaluations of fatigue, performance, or recognizer accuracy.
  • Future research should integrate recognizer performance with ergonomic criteria, conduct longitudinal studies in real-world settings, and explore cultural and generational diversity in gesture design.

Summary

This study reimagines gesture design for lightweight AR glasses by integrating physical therapists as expert diagnosticians. It introduces the Everyday-AR Golden Ergonomic Canvas, a chest-level, elbow-anchored workspace for sustainable gestures, and a stage-aware framework linking ergonomic stability with social legibility across six everyday contexts. The findings highlight the need to prioritize sustainability over naturalness, offering a recognizer-agnostic reference frame for designing gestures that balance fatigue resistance, social coherence, and usability. Future work will extend these insights into quantitative evaluations, longitudinal studies, and culturally diverse contexts.