Designing with Multi-Agent Generative AI: Insights from Industry Early Adopters

In this paper we present the results of our investigation into how employees at Microsoft, as early adopters of multi-agent generative AI systems, navigate the complexities of designing, testing, and deploying these technologies to extend the organization's product ecosystem. Through interviews with thirteen developers, we uncover the challenges, use cases, and lessons when designing with and for multi-agent AI frameworks. Our analysis reveals how participants leveraged this advanced emerging technology to enhance collaboration, productivity, customer support, creative processes, and security. Key design strategies include managing agent complexity, fostering transparency, and balancing agent autonomy with human oversight, essential considerations for human-agent interaction design. We provide empirical insights into the capabilities and limitations of multi-agent systems in real-world contexts, informing the design of future AI systems that align AI capabilities with human-centered design. By emphasizing first-person experiences and strategies, our research bridges human needs and AI potentials, advancing both the practice and theory of designing with and for AI systems.