ThrowIO: Actuated TUIs that Facilitate "Throwing and Catching" Spatial Interaction with Overhanging Mobile Wheeled Robots

Title of the Paper

ThrowIO: Actuated TUIs that Facilitate "Throwing and Catching" Spatial Interaction with Overhanging Mobile Wheeled Robots

Paper Information

• Subject Area: Human-Computer Interaction and Robotics, exploring a novel actuated tangible user interface for spatial interaction.
• Keywords: Actuated Tangible User Interface, Human-Robot Interaction, Human-Computer Interaction, Swarm User Interface, Spatial Interaction

Research Background and Problem

• Challenges Identified by the Authors:
  • Existing Virtual Reality (VR) and Augmented Reality (AR) interaction technologies lack support for real-world object throwing and catching actions. Many studies focus on providing haptic feedback or simulating hovering objects but rarely address highly interactive "throwing and catching" operations.
  • Throwing and catching are natural actions in daily activities, such as trash disposal, sports training, and parent-child interaction. However, current technologies struggle to preserve the authentic tactile sensation and learning experience of these interactions.
• Significance:
  • Throwing and catching are critical spatial interaction behaviors used in motor skill learning, immersive experiences, and entertainment. Developing new technologies to enhance such interactions not only benefits learning and gaming but also extends to practical scenarios like storage and communication.
• Motivation and Related Work:
  • Previous research on robotic manipulation of throwing and catching objects often requires complex physical setups and rapid response control. These systems typically overlook direct spatial interaction with users.
  • Existing solutions in virtual reality fail to simulate the tactile sensation of physical object throwing and catching, as well as the act of performing these operations.

Solution

• Proposed Method and Innovation:

  • Core Concept: The authors propose a novel actuated tangible user interface (TUI) called "ThrowIO," which uses ceiling-mounted mobile wheeled robots as a medium to facilitate real-world object throwing and catching interactions.
  • Implementation: Users throw magnetized objects onto a ferromagnetic ceiling surface, where robots move or drop the objects to designated positions for retrieval.
  • Innovation: Combines physical object interaction with digital system support, showcasing its potential in immersive experiences, learning scenarios, and everyday spatial applications. ThrowIO is easy to install and integrates seamlessly into daily environments.

• Implementation Steps and Technology:

  • Utilized mobile wheeled robots (toio) with custom mechanical shells to enable moving and dropping actions.
  • Employed a ferromagnetic surface combined with an RGBD camera to track the location of thrown objects, with algorithms analyzing object trajectories.
  • The system incorporates graphical augmented interfaces, such as screen displays and projection mapping, to enhance user experience and interactivity.

Research Outcomes

• Specific Results:

  • Developed a prototype system demonstrating the feasibility of robot-controlled throwing and catching of objects on a suspended surface.
  • Realized multiple application scenarios, including motor skill training, interactive gaming, immersive experiences, ceiling storage, and communication.
  • Technical Evaluation:
    • The average success rate of objects adhering to the suspended surface after being thrown was 79.48%, while the success rate for retrieving objects was 90.71%.
    • The reliability of robot operations for moving and dropping objects was high, achieving a 100% success rate.
  • User Study:
    • Tested with 16 participants, who provided positive feedback on the system's usability, intuitiveness, and practicality, particularly praising its applications in gaming and storage scenarios.
    • Suggested improvements, such as enhancing adhesion performance and adding visual cues for state detection.

• Advantages and Limitations:

  • Advantages: Integrates physical and digital interactions, providing users with a realistic tactile experience. ThrowIO is easy to install and supports various height adjustments and application extensions.
  • Limitations:
    • Objects thrown at certain angles and speeds may not adhere well to the surface.
    • The current prototype is limited by the number of robots and the tracking system, preventing simultaneous interaction with multiple objects.
    • The robot's dropping speed and angle are fixed, limiting precise control in immersive experiences.

• Future Directions:

  • Develop scalable tracking technologies to support simultaneous operation of multiple robots.
  • Optimize the material and shape of thrown objects to enhance adhesion performance.
  • Improve the flexibility of robot dropping actions to allow control over drop angles and speeds.
  • Expand system applications to surfaces like walls and windows, while supporting multi-user interactions.

Conclusion

ThrowIO reveals the potential of a novel spatial interaction technology by combining the tactile sensation of real objects with digital system support. It offers innovative solutions for motor skill training, entertainment, object storage, and communication. Despite challenges in performance and scalability, it opens a unique direction for future spatial user interface design.