Embedded vs. Situated: An Evaluation of AR Facial Training Feedback

Paper Title

Embedded vs. Situated: An Evaluation of AR Facial Training Feedback

Publication Info

• Topic area: Augmented reality (AR) feedback for facial muscle training
• Keywords: augmented reality, facial exercises, embedded feedback, situated visualization, cognitive load, user experience, motor learning, ARCore, muscle activation, real-time feedback

Background and Problem

• Problem / challenge: Existing facial muscle training systems rely on visual feedback that is spatially disconnected from the face, which hinders interpretability and effectiveness. While embedded AR feedback has proven effective for gross motor training, its application to fine-grained facial movements remains underexplored.
• Significance: Effective facial muscle training has applications in rehabilitation (e.g., facial paralysis), performance training (e.g., actors, public speakers), and motor skill acquisition. Understanding how feedback placement affects performance and user experience is critical for designing better systems.
• Motivation and related work: Prior AR-based systems for motor learning have demonstrated the benefits of spatially aligned feedback for gross motor tasks. However, facial training systems predominantly use situated feedback, which introduces cognitive challenges. This paper addresses the gap by evaluating embedded feedback for facial exercises.

Solution

• Proposed approach: The study evaluates three AR feedback conditions—ARSelfie (embedded), Mannequin (proxy-embedded), and BarChart (situated)—against a no-feedback Baseline condition to assess their impact on performance, cognitive load, and user experience during facial muscle training.
• Novelty:
  1. Extension of embedded AR feedback principles from gross motor to fine-grained facial exercises.
  2. Empirical evidence on the trade-offs between embedded and situated feedback for facial muscle training.
  3. Design guidelines for balancing performance accuracy, feedback interpretability, and user comfort in AR-based systems.
• Procedure and key techniques:
  1. Developed a mobile AR application using Google ARCore to track facial landmarks and estimate muscle activation in real-time.
  2. Designed four visualization conditions: ARSelfie, Mannequin, BarChart, and Baseline.
  3. Conducted a within-subjects study (N = 24) where participants performed three facial exercises (smile, eyebrow raise, reverse frown) across all conditions.
  4. Measured exercise accuracy, cognitive load (NASA-TLX, extraneous cognitive load), and user experience (UEQ).

Results

• Concrete findings:
  • BarChart achieved the highest accuracy (mean = 16.88) but imposed the highest cognitive load.
  • ARSelfie and Mannequin reduced cognitive load (ECL: 6.50 and 7.71, respectively) and were rated higher in user experience.
  • ARSelfie was the most preferred condition, followed by Mannequin, BarChart, and Baseline.
• Advantage over baselines:
  • Embedded feedback (ARSelfie and Mannequin) reduced cognitive effort and improved user experience compared to BarChart and Baseline.
  • Situated feedback (BarChart) supported higher accuracy but required more cognitive effort due to attention shifts.
• Experiments / evaluation:
  • Participants performed six cycles of three facial exercises under four feedback conditions.
  • Metrics included accuracy, time to first optimal repetition, number of optimal repetitions, NASA-TLX workload, UEQ ratings, and ECL scores.
  • Results were analyzed using repeated measures ANOVA and thematic qualitative analysis.
• Limitations and future work:
  • Short-term study in a controlled lab setting; long-term learning and real-world applicability remain untested.
  • Feedback system relies on heuristic-based muscle activation estimates; validation with physiological measurements (e.g., EMG) is needed.
  • Small, homogeneous participant sample limits generalizability; future studies should include diverse populations and clinical groups.

Summary

This study investigates the impact of spatial feedback placement on facial muscle training using AR. Embedded feedback (ARSelfie and Mannequin) reduced cognitive load and enhanced user experience compared to situated feedback (BarChart), though BarChart achieved higher accuracy. The findings suggest that spatially aligned feedback principles from gross motor training can be effectively applied to facial exercises. Design recommendations include combining spatial alignment with explicit magnitude indicators and offering customizable self-representation options. These insights have implications for rehabilitation, performance training, and motor skill acquisition.