Building Human–Multi-Agent Teams for Creative Works

Paper Title

Understanding Human–Multi-Agent Team Formation for Creative Work

Publication Info

• Topic area: Human–AI collaboration in creative workflows
• Keywords: Human–Multi-Agent Teams, generative AI, team formation, creative ideation, human orchestration, multi-agent systems, shared mental models, role allocation, design ideation, human-centered AI

Background and Problem

• Problem / challenge: While generative AI enables collaboration with multiple AI agents in distinct roles, the formation of effective Human–Multi-Agent Teams (HMATs) remains underexplored. Challenges include inter-agent interaction complexity, unexpected behaviors, and unclear functional boundaries.
• Significance: Effective HMAT formation could enhance creative workflows by leveraging diverse perspectives and decomposing complex tasks. This is particularly relevant for addressing wicked problems in creative industries.
• Motivation and related work: Prior research has focused on Human-Agent Teams (HATs) and Multi-Agent Systems (MAS), but most studies assume fixed team configurations or fully autonomous pipelines. These approaches often fail to integrate human oversight and iterative refinement, which are essential for creative work. This paper addresses the gap by exploring how humans can form and orchestrate HMATs.

Solution

• Proposed approach: CrafTeam, a technology probe that enables users to form and collaborate with HMATs, focusing on five key dimensions of team formation: team size, structure, role allocation, member composition, and shared mental models.
• Novelty:
  1. Development of CrafTeam to explore HMAT formation in creative ideation tasks.
  2. Empirical insights from a user study with 12 design practitioners iteratively forming and refining HMATs.
  3. Identification of design considerations for human-orchestrated HMATs, emphasizing multi-party communication and progressive team evolution.
• Procedure and key techniques:
  • Users iteratively form HMATs by configuring five dimensions via CrafTeam’s interface.
  • Teams engage in ideation tasks, followed by reflection phases to evaluate team effectiveness.
  • AI agents operate autonomously within user-defined roles, supported by profiling and memory modules for dynamic interaction.

Results

• Concrete findings:
  • Participants initially formed autonomous teams but shifted to human-orchestrated formations due to AI agents’ inability to make value judgments or set clear directions.
  • Teams averaged 4.61 members, with participants favoring single-tier hierarchies where they acted as leaders.
  • AI agents generated 451 ideas, while users primarily evaluated ideas and provided feedback.
  • Participants emphasized role specialization and diversified agent personas to improve team performance.
• Advantage over baselines:
  • Unlike traditional HATs or MAS, CrafTeam supports iterative team refinement, allowing users to adapt team configurations based on real-time feedback and outcomes.
  • Human-orchestrated HMATs demonstrated better alignment with creative goals compared to fully autonomous setups.
• Experiments / evaluation:
  • A three-hour study with 12 design practitioners, each completing three cycles of team formation, ideation, and reflection.
  • Metrics included team size, structure, role allocation, member composition, shared mental models, and ideation outputs.
• Limitations and future work:
  • Limited sample size and short study duration; no longitudinal evaluation.
  • Focused on ideation tasks; findings may not generalize to other workflows.
  • Did not explore multi-human HMATs or ethical implications in depth.
  • Future work should investigate diverse creative domains, multi-human scenarios, and methods to balance human agency with AI assistance.

Summary

This study introduces CrafTeam, a system enabling users to form and collaborate with Human–Multi-Agent Teams (HMATs) for creative ideation tasks. Through a user study with 12 design practitioners, participants iteratively refined their teams, shifting from autonomous to human-orchestrated formations due to AI agents’ limitations in making value judgments. Findings highlight the importance of human leadership, role specialization, and diversified agent personas in HMAT formation. The study underscores the need for scalable multi-party communication and iterative team evolution to reduce user burden while maintaining creative alignment. These insights provide a foundation for designing effective human-orchestrated multi-agent teams in creative workflows.