Figaro: A Tabletop Authoring Environment for Human-Robot Interaction

Document Title

Figaro: A Tabletop Authoring Environment for Human-Robot Interaction

Document Information

• Research Area: Human-Computer Interaction Design and Robot Programming
• Keywords: Human-Computer Interaction Design, Tabletop Interface, Robot Programming, Shadow Puppetry Technique, Program Synthesis, Visual Programming

Research Background and Issues

• Identified Problems or Challenges:

  1. In social robot programming, designers need to balance between operational and programming abstractions. Existing tools fail to adequately support designers in creating interactions within the context of target scenarios.
  2. Current programming tools are either too low-level or overly abstract, requiring designers to construct complex mental models to bridge abstract syntax and real-world scenarios.
  3. There is a lack of clear mapping between program design and actual deployment, often necessitating extensive simulation and testing by designers.

• Significance:

  1. Providing intuitive development tools for human-computer interaction designers can accelerate the design and deployment process.
  2. Reducing learning barriers in robot programming, particularly for non-professionals and newcomers.

• Research Motivation and Related Work:

  1. Inspired by shadow puppetry techniques, the authors propose Figaro, a tabletop system that tightly integrates social interaction design with real-world environments, allowing designers to focus on behavior specification.
  2. Introducing a system that transforms design demonstrations into high-level programs, addressing the gap between interaction design abstraction and scenario implementation.

Solution

• Proposed Solution:

  1. Designed and implemented a tabletop projection-based visual programming tool, Figaro, leveraging shadow puppetry techniques to enable designers to program by demonstrating interaction scenarios.
  2. The Figaro system allows users to demonstrate scenarios using physical objects and voice commands, employing real-time program synthesis technology to convert these demonstrations into complete robot interaction programs.

• Innovations:

  1. Figaro employs virtual tabletop projection technology to embed the design process within a miniature representation of the actual interaction environment, enhancing design intuitiveness and immersion.
  2. Integrates real-time voice recognition and classification systems, enabling designers to specify human and robot behaviors during demonstrations.
  3. Utilizes program synthesis technology to automatically convert multi-scenario demonstration results into complete robot programs, supporting modular interruption mechanisms.

• Implementation Steps and Techniques:

  1. Provide a projection tabletop displaying the physical layout of the target environment.
  2. Users demonstrate scenarios using instrumented puppets and describe robot behaviors through voice commands.
  3. Figaro records these demonstrations (including positions, actions, and speech) and converts them into execution trajectories.
  4. The system automatically generates interaction logic through program synthesis technology and deploys it to robots.

Research Outcomes

• Specific Outcomes:

  1. Developed the Figaro tool, enabling users to create complex human-robot interaction scenarios through intuitive demonstrations.
  2. Provided an open-source implementation supporting collaborative design and multi-task logic workflows.

• Advantages Over Existing Solutions:

  1. Figaro reduces the learning curve for non-professional designers through an intuitive user interface.
  2. It is closer to real-world interaction contexts compared to traditional programming tools, avoiding cumbersome symbolic and memory-intensive processes.
  3. Automated program synthesis minimizes technical details in the design process, enhancing rapid iteration capabilities.

• Experimental or Evaluation Results:

  1. User studies indicate that Figaro effectively supports users in exploring interaction design and programming workflows.
  2. Most participants found the tool easy to use and helpful in generating new design ideas.

• Limitations and Future Directions:

  • Limitations:

    1. Currently supports only single-scenario demonstrations between two interaction roles (human and robot).
    2. Program synthesis is limited to user-provided demonstration cases and cannot handle unexplored paths or areas.
    3. The system struggles with high-precision ground positioning requirements in complex environments.

  • Future Directions:

    1. Expand support for multi-role interactions, such as group interaction scenarios.
    2. Enhance the generalization capability of program synthesis algorithms to learn unexplored states or paths.
    3. Integrate more advanced debugging modes to support design iteration and feedback optimization.
    4. Explore the combination of augmented reality technology with Figaro to enable real-time design adjustments in physical environments.

Conclusion

As a visual design tool, Figaro provides human-computer interaction designers and developers with a natural and intuitive way to create complex interaction logic. Inspired by shadow puppetry techniques, it delivers a low-threshold, demonstrable programming experience through a tabletop interface. This work highlights the significant potential of tabletop interaction and program synthesis technologies in robot interaction design.