AR-Enhanced Workouts: Exploring Visual Cues for At-Home Workout Videos in AR Environment

Title of the Paper

AR-Enhanced Workouts: Exploring Visual Cues for At-Home Workout Videos in AR Environment

Paper Information

• Domain: Application of Augmented Reality (AR) in at-home workout videos
• Keywords: AR, visual cues, at-home workouts, motor learning, real-time feedback, virtual human, 3D pose tracking, immersive learning, exercise science
• Conference: The 36th Annual ACM Symposium on User Interface Software and Technology (UIST ’23)
• Publication Date: October 29–November 1, 2023

Research Background and Problem

• Issues or Challenges:

  • At-home workout videos lack dynamic viewing angles, with fixed-perspective videos limiting users' understanding of three-dimensional body movements.
  • The absence of real-time feedback during exercises affects the accuracy and safety of movements.
  • Certain workout postures (e.g., prone or supine positions) may lead to posture errors when following video instructions.

• Significance:

  • At-home workouts are becoming increasingly popular due to their convenience, safety, and privacy.
  • Incorrect exercise postures can reduce workout effectiveness and even cause physical injuries.
  • AR technology has the potential to address these issues, offering a new user experience and learning method for home fitness.

• Research Motivation and Related Work:

  • Existing studies have explored the use of visual cues and immersive technologies (e.g., AR/VR) but have not fully investigated the design space and user acceptance of these technologies in exercise training.
  • The research aims to design an integrated AR system and provide immersive support for motor learning through iterative design.

Solution

• Main Approach:

  • Proposed an AR-based at-home workout system combining head-mounted display devices (HoloLens 2) and external cameras, utilizing 3D pose tracking technology to record and provide feedback on users' movements.
  • Designed a design space for three-dimensional visual cues, including directional guidance, measurement data, skeletal joint visualization, and metaphorical cues.

• Innovations:

  • Created a systematic design space for visual cues in 3D environments, specifying data types, task purposes, and application scenarios.
  • Provided real-time dynamic feedback to help users adjust their movements, addressing the limitations of traditional pre-recorded videos.
  • Validated the advantages of AR-enhanced workouts in improving movement understanding and memory retention through user testing.

• Implementation Steps and Technologies:

  1. Data Preparation:
     • Used Mixamo to create standard motion animations for virtual humans and MVN Link to capture missing movements.
  2. AR Environment Display:
     • Developed the application using Unity, capturing user skeletal data with Mediapipe to generate interactive AR visual cues.
  3. Visual Feedback Generation:
     • Compared user movements with standard motions, generating red and orange visual cues to guide users in real-time movement correction.
  4. Experiment and Interaction Design:
     • Provided a virtual control panel to support workout selection and adjustment of viewing angles.

Research Outcomes

• Specific Results:

  • Developed an AR-enhanced workout system integrating real-time pose tracking and visual cues.
  • Conducted two-stage user experiments to validate the effectiveness of visual cues and their benefits for movement understanding, memory, and practice.

• Comparison with Existing Solutions:

  • AR technology allows users to observe movements from multiple angles while keeping the target scene in the center of their field of view, overcoming the limitations of fixed-perspective videos.
  • Compared to 2D video methods, the 3D-enhanced approach increases immersion and improves users' grasp of movement details.

• Experimental or Evaluation Results:

  • Users generally found 3D displays more flexible and easier to understand and remember than 2D videos.
  • Metaphorical visual cues enhanced interaction enjoyment and memorability, although some complex designs increased cognitive load.
  • Real-time visual feedback positively supported users in adjusting their movements, though issues of delay and cognitive load were noted.

• Limitations and Future Directions:

  • Device Limitations: The weight and field-of-view constraints of current AR devices (e.g., HoloLens 2) affect certain workout movements.
  • Visual Cue Design: Feedback methods need further optimization to reduce users' cognitive load.
  • Future Research Directions:
    • Develop feedback mechanisms based on personalized body data and fitness levels.
    • Explore design optimizations for AR visual cues in dynamic adjustments and physical environment adaptation.
    • Further quantify the impact of latency on motor learning outcomes while introducing more efficient low-latency devices.