AvatARoid: A Motion-Mapped AR Overlay to Bridge the Embodiment Gap Between Robots and Teleoperators in Robot-Mediated Telepresence

Research Background and Issues

• Identified Problems or Challenges: The authors explore how robotic telepresence systems can support social interactions between remote operators and on-site users. However, inconsistencies between the robot's form and the operator's representation lead to a "representation gap" in the on-site user's perception, negatively affecting interaction quality.
• Significance: In contexts such as healthcare, education, and workplace collaboration, telepresence systems enhance remote social interactions. However, current systems fail to effectively connect the operator with the robot in terms of embodiment, potentially causing negative impressions such as "bullying behavior" or violations of personal space.
• Research Motivation and Related Work: Although video displays on tablets and augmented reality (AR) 3D virtual avatars are widely used, video fails to establish a strong connection between the operator and the robot, while AR avatars may create overly high expectations for interaction among on-site users. The limitations of existing solutions necessitate a new approach to address this "embodiment gap."

Solution

• Proposed Method or Solution: The authors introduce a novel system—AvatARoid, which maps the remote operator's actions in real-time to an augmented reality (AR) avatar overlaid on a humanoid robot, aiming to bridge the embodiment gap between the operator and on-site users.
• Innovations: AvatARoid combines augmented reality technology with humanoid robots, synchronizing the operator's actions with the robot's movements in real-time. Additionally, it utilizes point cloud technology to generate highly dynamic AR avatars, enriching non-verbal interactions such as facial expressions and gestures.
• Implementation Steps and Key Technologies:
  1. Capturing the operator's motion data using RGBD sensors.
  2. Applying inverse kinematics (IK) algorithms to synchronize robot movements, ensuring smooth mapping of the operator's actions to the robot.
  3. Using point cloud technology to generate highly dynamic AR avatars and overlay them on the robot's upper body in real-time.
  4. Displaying the integrated virtual robot and operator avatar through AR head-mounted devices, enhancing on-site user perception.

Research Outcomes

• Specific Results:
  • Compared to traditional video representation, AvatARoid significantly improved on-site users' sense of embodiment of the operator.
  • The system strengthened the control connection between the operator and the robot, allowing on-site users to perceive high synchronization between the robot's movements and the operator.
  • In experiments, AvatARoid enhanced spatial co-presence and enriched non-verbal communication.
• Advantages: Compared to existing systems based on video or single robot representation, AvatARoid excels in non-verbal expression and spatial perception, making it easier for on-site users to accept the robot as the operator's embodiment.
• Experimental and Evaluation Results:
  • Experiments showed that, compared to baseline conditions (robot-only) and video conditions, AvatARoid significantly enhanced on-site users' recognition of spatial co-presence and control.
  • Interviews revealed that AvatARoid provided on-site users with a more humanized and emotionally connected interaction experience.
• Limitations and Future Directions:
  • The current system supports only upper-body point cloud avatars, and users noted that the lack of full-body embodiment limits realism.
  • The naturalness and dynamic synchronization of robot movements require further improvement.
  • Issues such as technical latency and device resolution affect the fluidity of real-time interactions.
  • Future research should incorporate real-world robots to further validate the system's effectiveness in enhancing social presence.

Conclusion and Outlook

The AvatARoid system significantly improves embodiment representation and social interaction quality by overlaying dynamic AR avatars of remote operators onto humanoid robots. Overcoming technical bottlenecks and conducting more experiments in real-world robotic scenarios will further advance the application potential of combining augmented reality and robotic technologies.