RobotAR: An Augmented Reality Compatible Teleconsulting Robotics Toolkit for Augmented Makerspaces Experiences

Title of the Paper

RobotAR: An Augmented Reality Compatible Teleconsulting Robotics Toolkit for Augmented Makerspace Experiences

Paper Information

• Subject Area: Integration of robotics in remote education and augmented reality (AR) applications
• Keywords: robotics, augmented reality, teleconsulting, virtual makerspace, voice assistant, educational tools, circuit design, user research, usability evaluation

Research Background and Problem

• Problem or Challenge: With the rise of online education, especially during the pandemic, remote teaching of hands-on courses (e.g., engineering and makerspace activities) faces significant challenges, including reduced student engagement, increased complexity in diagnosing issues, and difficulty in providing real-time feedback.
• Importance: Makerspace-style teaching relies heavily on instructor support, troubleshooting, and demonstration-based learning. Such interaction and hands-on practice are critical for fostering students' creativity and technical skills.
• Research Motivation and Related Work: Existing remote teaching platforms (e.g., Zoom) provide real-time interaction capabilities but lack the ability to explore specific problem areas and deliver immersive teaching experiences in hands-on environments. Additionally, while social and telepresence robots show educational potential, their adoption is limited by high hardware costs and installation complexity.

Solution

• Method or Solution:

  • Designed and developed a toolkit named RobotAR, an augmented reality-compatible teleconsulting robotics tool.
  • RobotAR integrates augmented reality technology, enabling instructors to interact, annotate, and provide visual guidance via robots in real time.
  • Includes an AI voice assistant to reduce repetitive tasks during hands-on experiments.
  • The toolkit adapts to students' personal physical workbench environments and supports hybrid virtual/physical operations.

• Innovations:

  • Combines AR technology with robotic control for remote teaching, allowing instructors to add three-dimensional (3D) spatial information, including real-time annotations, prompts, and demonstration examples.
  • Introduces a voice assistant as the primary support for addressing students' common questions.
  • RobotAR's multi-degree-of-freedom mobility enables precise positioning, facilitating attention to specific issues in the workspace.
  • Improves upon the limitations of existing Zoom-based models, with hardware enabling flexible desktop-based movement.

• Implementation Steps:

  1. Hardware Design:
     • Robot chassis based on microprocessors, supporting multi-directional movement.
     • Adjustable smartphone holder for AR content display.
     • Utilizes students' own smartphones as software carriers.
  2. Software Development:
     • Developed a network architecture using Unity 3D for real-time video streaming and command relaying.
     • Leveraged ARCore for 3D spatial mapping to support content overlay.
     • Designed user interfaces (UI) for both student and instructor ends to facilitate interaction.
  3. AI Voice Assistant:
     • Trained the assistant using natural language processing frameworks (e.g., Wit.ai) to answer students' common questions.
     • Employed an iterative training mechanism to continuously update the AI database.

Research Outcomes

• Specific Results:

  • Compared to traditional Zoom-based teaching models, RobotAR demonstrated superior performance across multiple key educational metrics and user experience evaluations:
    • Students using RobotAR showed significant improvement in knowledge and skills related to current, voltage, circuit connections, and component functionality.
    • RobotAR significantly enhanced instructors' management capabilities and teaching effectiveness in remote environments.
    • Provided a more immersive remote learning experience for students, including the use of AR technology to visualize complex circuit concepts and real-time annotations.

• Advantages Over Existing Solutions:

  • Compared to traditional Zoom meetings and other static teleconsulting solutions, RobotAR enhances teaching effectiveness through AR and dynamic interaction.
  • Offers instructors and students a more intuitive and clear spatial and information delivery model.
  • The voice assistant and robotic autonomous movement features reduce interruptions caused by students needing to adjust equipment during experiments.

• Experimental or Evaluation Results:

  • Comparative studies between RobotAR and Zoom demonstrated that RobotAR significantly improved performance in 6 out of 7 key competencies.
  • User surveys revealed that RobotAR outperformed traditional Zoom models in instructor presence, effective real-time guidance, and teleconsulting quality.
  • The voice assistant enhanced students' independent learning capabilities, while AR supported repeated access to learning content.

• Limitations and Future Directions:

  • The current system limits simultaneous one-to-many teleconsulting sessions.
  • The robot lacks obstacle avoidance functionality and offers limited support for multi-angle camera switching.
  • Future improvements include adding broadcast options to support multi-student remote teaching, enhancing the AI voice assistant's Q&A capabilities, and upgrading hardware with automated protection features.

Conclusion

This paper demonstrates the educational potential of combining robotics and augmented reality through the application of the RobotAR toolkit in remote makerspace environments. RobotAR not only improves instructors' guidance capabilities but also optimizes virtual interactions and enhances students' experimental experiences in remote settings.