Out of Control: Effects of Multimodal Self-similarity on Embodiment During Autonomous Avatar Demonstrations in Virtual Reality

Paper Title

Out of Control: Effects of Multimodal Self-similarity on Embodiment During Autonomous Avatar Demonstrations in Virtual Reality

Publication Info

• Topic area: Interaction of avatar self-similarity and autonomy in virtual reality embodiment.
• Keywords: Virtual reality, embodiment, self-similarity, avatar autonomy, user experience, motor learning, multimodal interaction, head-direction tracking, presence, frustration.

Background and Problem

• Problem / challenge: Autonomous avatar demonstrations in VR training and rehabilitation reduce user control, which can disrupt embodiment. Prior research has not systematically examined how multimodal self-similarity (appearance and voice) might mitigate this loss of embodiment.
• Significance: Embodiment is crucial for motor learning, engagement, and rehabilitation. Understanding how to preserve embodiment during autonomous phases can improve VR systems for these applications.
• Motivation and related work: Previous studies show that self-similar avatars enhance embodiment and identification, but the effects of combining self-similarity with autonomy remain unclear. This study builds on findings about visual and motor congruence, agency, and observational learning to address this gap.

Solution

• Proposed approach: Investigate how multimodal self-similarity (visual and vocal) and autonomy affect embodiment, user experience, and behavioral responses in VR using a 2 × 2 factorial design.
• Novelty:
  1. Examines the interaction of self-similarity and autonomy in VR, focusing on embodiment and user experience.
  2. Introduces multimodal self-similarity (visual and vocal) to study its effects during autonomous avatar demonstrations.
  3. Combines subjective measures (e.g., presence, agency) with objective behavioral data (head-direction tracking).
  4. Provides design principles for balancing autonomy and personalization in VR systems.
• Procedure and key techniques:
  • Participants (N = 24) embodied avatars in four experimental conditions: self-similar/non-self-similar appearance and voice × autonomous/non-autonomous behavior.
  • Self-similar avatars were created using participant photos and voice recordings, while non-self-similar avatars were generic but sex-matched.
  • Participants performed a block-assembling task, with head-direction tracking used as a proxy for gaze.
  • Self-reported measures included presence, agency, frustration, and self-attribution.

Results

• Concrete findings:
  • Self-similarity increased body location ratings (p = .018, ηp² = .221), presence (p = .008, ηp² = .265), and self-attribution (p < .001, ηp² = .623).
  • Autonomy reduced agency (p < .001, ηp² = .709), presence (p < .001, ηp² = .427), and self-attribution (p < .001, ηp² = .503) but increased emotional reactivity (p < .001, ηp² = .500) and frustration (p = .012, ηp² = .246).
  • Head-direction patterns showed increased mirror focus in autonomous conditions (p < .001, ηp² = .698) and task focus in non-autonomous conditions (p = .001, ηp² = .386).
• Advantage over baselines:
  • Self-similar avatars preserved presence and self-attribution during autonomous phases, mitigating some embodiment loss.
  • Behavioral data revealed distinct attention patterns, supporting observational learning in autonomous conditions.
• Experiments / evaluation:
  • Study design: 2 × 2 within-group factorial design.
  • Metrics: Self-reported embodiment and user experience measures, head-direction tracking.
  • Datasets: 24 participants, balanced for sex and age.
• Limitations and future work:
  • Head-direction tracking lacked precision compared to eye-tracking.
  • Avatar and voice fidelity were constrained by current tools.
  • Limited task scope (block assembly) and sample diversity.
  • Future work should explore multimodal autonomy, diverse tasks, and long-term learning effects.

Summary

This study investigates how multimodal self-similarity (appearance and voice) and autonomy affect embodiment, user experience, and behavior in VR. Self-similarity enhanced presence, body location, and self-attribution, while autonomy reduced agency but increased emotional reactivity and frustration. Behavioral data revealed distinct attention patterns, with autonomous conditions promoting mirror focus and non-autonomous conditions encouraging task focus. These findings suggest that self-similar avatars can buffer embodiment loss during autonomous phases, offering design principles for VR systems in training and rehabilitation. Future research should address limitations in tracking precision, task diversity, and avatar fidelity.