HandAvatar: Embodying Non-Humanoid Virtual Avatars through Hands

Document Title

HandAvatar: Embodying Non-Humanoid Virtual Avatars through Hands

Document Information

• Subject Area: Human-Computer Interaction, Control of Non-Humanoid Virtual Avatars in Virtual Reality Environments
• Keywords: Virtual avatars, embodiment, mixed reality, gesture interaction, virtual reality, animation creation, body mapping, user experience, environmental interaction

Research Background and Problem Statement

• Identified Problems or Challenges:

  1. Users face limitations in controlling and operating non-humanoid avatars (e.g., spiders, crocodiles) in virtual reality. Traditional methods primarily rely on full-body movements or controller-based operations, but these approaches are constrained by differences in body structure and degrees of freedom, making precise real-time control difficult.
  2. Current mapping methods for non-humanoid avatars often require large-scale body movements, which not only impose a high physical burden but also result in low control efficiency.

• Significance of the Research:

  1. Interaction with non-humanoid avatars expands user experience and operational possibilities in virtual reality environments, fostering creativity and imagination.
  2. Enhancing the precision and comfort of controlling non-humanoid avatars is crucial for animation production, educational entertainment, and virtual social interactions.

• Motivation and Related Work:

  1. Body mapping technologies are typically used for humanoid avatar control, while optimized solutions for non-humanoid avatars remain lacking.
  2. Digital puppetry and gesture interaction are relatively straightforward and convenient but are currently mainly applied in animation creation, with limited fine-tuned mapping designs for non-humanoid characters.
  3. Literature reviews indicate that the high flexibility and structural complexity of hands can aid in controlling non-humanoid avatars, but optimized hand-to-avatar mapping strategies are needed.

Proposed Solution

• Proposed Method: A novel "HandAvatar" technique is introduced, enabling users to control non-humanoid avatars through their hands while optimizing the mapping between hand joints and the avatar's key joints. The approach includes user studies, automated mapping algorithm design, and multi-task evaluations.

• Innovative Contributions:

  1. Optimized Algorithm: An optimized hand-to-avatar mapping algorithm that balances control precision, structural similarity, and user comfort.
  2. User-Driven Design: A data-driven approach based on user studies to uncover user preferences when designing hand-to-avatar mappings.
  3. Utilization of Hand Flexibility: Exploration of using hand control to perform complex non-humanoid actions as an alternative to full-body movements.

• Implementation Steps and Key Techniques:

  1. User Study: Observing users as they design gesture mappings for various non-humanoid avatars and analyzing preferences derived from user voting.
  2. Algorithm Design: Developing an optimization algorithm based on user study results, incorporating hand biomechanics and virtual avatar parameters as inputs.
  3. Multi-Task Evaluation: Assessing the effectiveness of HandAvatar in static posing, dynamic animation creation, and creative exploration tasks, and comparing it with existing methods.

Research Outcomes

• Specific Results:

  1. The HandAvatar technique significantly improved the control precision of non-humanoid avatars, reducing static pose deviation by 40% and dynamic animation joint deviation by 25% compared to full-body-based methods.
  2. The automated hand-to-avatar optimized mapping demonstrated excellent performance across control, comfort, and similarity dimensions.
  3. User surveys indicated that HandAvatar reduced physical burden while offering more refined control experiences.

• Comparison with Existing Solutions:

  1. Compared to full-body control methods such as KinÊtre, HandAvatar significantly reduced users' physical burden and improved control precision.
  2. HandAvatar showed notable advantages in controlling structurally complex avatars (e.g., spiders).

• Experimental or Evaluation Results:

  1. Manual Mapping vs. Algorithm-Generated Mapping: The optimized mappings generated by HandAvatar achieved an average score of 5.07/7, outperforming user-designed mappings (4.49/7).
  2. Demonstrations of application scenarios highlighted HandAvatar's potential in virtual social interactions, 3D animation creation, and scene design.

• Limitations and Future Directions:

  1. Currently, the system only supports one-to-one mapping between hand joints and avatar key nodes. Future work could explore one-to-many mappings or more complex degrees-of-freedom mapping methods.
  2. The optimization algorithm requires further improvement to support multi-character scenarios and tracking under complex occlusion conditions.
  3. Interactive methods incorporating environmental feedback and haptic design warrant further exploration.