Interactive Robotic Plastering: Augmented Interactive Design and Fabrication for On-site Robotic Plastering

Title of the Paper

Interactive Robotic Plastering: Augmented Interactive Design and Fabrication for On-site Robotic Plastering

Paper Information

• Research Domain: Human-Computer Interaction, Architectural Robotics, Augmented Reality, and Digital Fabrication
• Keywords: Interactive Fabrication, Augmented Reality, Robotics, Digital Fabrication, Demonstration Programming, Construction Automation, User Study, Full-scale Prototyping, Human-Robot Collaboration, Integrated Design and Manufacturing

Research Background and Issues

• Problems and Challenges:
  • The use of complex or flexible materials (such as plaster) in construction traditionally relies on craftsmanship and manual expertise, making full automation difficult.
  • Conventional robotic workflows primarily depend on pre-defined linear operational sequences, but the uncertainties of on-site construction environments increase the complexity of automation.
  • There is currently a lack of tools enabling designers and skilled workers to collaborate with machines in real-time for architectural fabrication.
• Significance:
  • Plaster spraying, as a critical construction process, can achieve more efficient and precise operations through robotics while reducing physical labor demands.
  • Integrating craftsmanship experience with robotic precision during construction helps explore new possibilities in design and fabrication.
• Research Motivation and Related Work:
  • Human-robot collaboration research mainly focuses on how humans directly influence physical fabrication outcomes through interaction.
  • Although some studies have been conducted in this direction, most are limited to small-scale scenarios or lack real-time user feedback.
  • Additionally, existing research has not fully utilized augmented reality to enhance user interaction with robots on construction sites.

Solution

• Main Methods and System:
  • Proposes an interactive robotic plastering system (IRoP) that integrates design tools, augmented reality interfaces, and robotic plastering systems.
  • The system generates robotic motion trajectories adapted to material and mechanical constraints based on user semantic input via a controller.
  • Adopts a "demonstration programming" approach, allowing users to generate complex robotic trajectories through direct controller manipulation.
• Innovations:
  • Achieves real-time integration of interactive design and robotic fabrication processes on construction sites.
  • Provides a projection-based augmented reality interface to help users evaluate design effects during the design process.
  • Breaks away from traditional CAD file-driven robotic fabrication workflows, enabling users to interact directly with robots at lower costs.
• Implementation Steps:
  1. Filter Selection: Allows users to select and define different style filters, mapping human actions to robotic trajectories.
  2. Design Mode: Users physically draw or adjust design sketches to complete plaster designs for wall surfaces.
  3. Spraying Execution: After submitting the design plan, the system generates motion trajectories, and the robot performs plaster spraying based on the trajectories.
  4. Feedback and Adjustment: Uses depth cameras to scan target surfaces, providing real-time feedback to adjust subsequent paths.
• Key Technologies:
  • Motion Tracking Technology: Utilizes HTC Vive motion capture devices for high-precision manual path input.
  • Augmented Reality Projection: Projects design information onto construction walls in real-time using projectors.
  • Computational Model: Builds an interactive 3D modeling environment using Python and Grasshopper to generate and adjust robotic trajectories in real-time.

Research Outcomes

• Specific Achievements:
  • Developed a dynamic interactive plaster fabrication system integrating computation, vision, and robotics.
  • Created multiple style filters, enabling users to generate artistic and highly complex plaster wall surfaces using different operational modes.
  • Designed a comprehensive on-site augmented user interface supporting human-robot collaboration, suitable for designers and skilled workers.
• Comparative Advantages:
  • Compared to traditional automation systems, IRoP allows users to make dynamic decisions and adjust processing paths based on actual working environments.
  • Provides a highly accessible robotic fabrication interface for users with varying technical backgrounds (designers and skilled workers).
• Experimental and Evaluation Results:
  1. Designer Participation Study:
     • In the experiment, 18 designers participated in a 110-square-meter indoor wall project.
     • Designers acknowledged the intuitive usability of the system and appreciated the real-time 1:1 scale design and full-process user involvement.
  2. Skilled Worker Experience:
     • Five professional plasterers quickly learned to use IRoP and completed trial tasks.
     • Users generally praised the system's "gamified" characteristics, describing it as easy to use, intuitive, and enjoyable.
• Limitations and Future Directions:
  • The current system's hardware still has spatial coverage limitations, requiring manual relocation of the robot.
  • Augmented reality projection is constrained by the device's viewing angle, suggesting future improvements with mobile projection equipment.
  • The system needs further functional expansion to adapt to more practical construction tasks and other material environments.
  • Proposed future directions include remote collaborative design, integration of customizable interactive devices, and extension to mobile robotic platforms.