Exploring Co-located Collaborative Visual Programming of Industrial Robotic Workplaces in Handheld AR

Paper Title

Exploring Co-located Collaborative Visual Programming of Industrial Robotic Workplaces in Handheld AR

Publication Info

• Topic area: Collaborative visual programming in industrial robotic workplaces using handheld augmented reality.
• Keywords: Collaborative programming, augmented reality, industrial robotics, visual programming, co-located collaboration, end-user programming, workspace awareness, handheld AR, human-computer interaction, industrial automation.

Background and Problem

• Problem / challenge: Existing end-user programming approaches focus on individual users and lack support for co-located collaboration in industrial robotic programming. Challenges include maintaining awareness, deictic referencing, and authorship attribution in shared AR environments.
• Significance: Industrial robotic workplaces require collaboration between domain experts with complementary knowledge. Supporting co-located programming in AR can improve usability, coordination, and correctness in configuring robotic systems.
• Motivation and related work: Prior work has explored collaborative programming, AR-based programming, and collaborative AR but has not addressed their intersection in industrial contexts. Challenges like spatial anchoring, coordination, and shared awareness remain underexplored in such settings.

Solution

• Proposed approach: A handheld AR system for co-located collaborative visual programming of industrial robotic workplaces, enabling users to define and edit program logic directly in the physical environment.
• Novelty:
  1. Empirical evaluation of spatial visual programming for robotic workplaces in a co-located AR setting.
  2. Identification of collaboration patterns, coordination strategies, and challenges in awareness and communication.
  3. Design implications for improving collaborative AR programming systems in industrial contexts.
• Procedure and key techniques:
  • The system uses spatially anchored actions, markerless tracking, and synchronized multi-user sessions via a central ARServer.
  • Actions are represented as visual elements in AR, allowing direct programming in the physical workspace.
  • Features include an object-locking mechanism, synchronized updates, and Python code generation for extensibility.
  • The study involved five participant pairs collaboratively programming a robotic workplace using the system.

Results

• Concrete findings:
  • All participant pairs successfully completed a non-trivial programming task (approximately 20 steps) within ~50 minutes.
  • Collaboration styles varied, with sequential work dominating but interspersed with parallel episodes.
  • Participants rated collaboration support highly (mean: 4.2/5) despite challenges in awareness and authorship attribution.
• Advantage over baselines: No direct baseline comparison was conducted, but the system demonstrated feasibility and usability for co-located collaborative programming in AR, a previously unexplored area.
• Experiments / evaluation:
  • Conducted with 10 participants (5 pairs) in a controlled lab setting using a mock industrial robotic workplace.
  • Tasks involved programming robots and conveyors to handle quality control and material flow.
  • Data collected through video recordings, questionnaires, and qualitative coding of collaboration behaviors.
• Limitations and future work:
  • Small sample size and controlled lab setting limit generalizability.
  • Participants were mostly IT students with limited industrial experience.
  • Future work should involve industry practitioners, explore asynchronous collaboration, and evaluate additional coordination features like virtual pointers and authorship indicators.

Summary

This study introduces a handheld AR system for co-located collaborative visual programming of industrial robotic workplaces, enabling users with complementary expertise to jointly configure robotic systems. An empirical evaluation with five participant pairs demonstrated the system's feasibility and identified challenges in awareness, coordination, and referencing. Participants successfully completed a non-trivial programming task, highlighting the potential of AR for industrial collaboration. Future research will focus on enhancing coordination mechanisms, supporting asynchronous collaboration, and involving industry practitioners to validate the system in real-world scenarios.