Collaborative Document Editing with Multiple Users and AI Agents

Paper Title

Collaborative Document Editing with Multiple Users and AI Agents

Publication Info

• Topic area: Integration of AI agents into collaborative writing environments.
• Keywords: Collaborative writing, AI agents, team-AI interaction, document editing, shared resources, agent profiles, task delegation, human-centered design, large language models, mixed-initiative systems.

Background and Problem

• Problem / challenge: Current AI writing tools are designed for individual use, requiring users to leave collaborative environments to interact with AI. This creates challenges in reintegrating AI outputs into shared documents and lacks support for team-based workflows.
• Significance: Addressing this gap is critical to enhancing collaborative writing processes, meeting industry demands for AI integration in shared tools, and supporting human-centered teamwork with AI.
• Motivation and related work: Prior research has focused on either collaborative writing among humans or individual writing with AI, neglecting the intersection. Existing tools lack UI patterns for integrating AI into team workflows. This paper builds on gaps identified in surveys and studies, such as the absence of AI-assisted tools targeting collaborative writing.

Solution

• Proposed approach: A prototype system integrating customizable AI agents into collaborative document editors, enabling shared use through agent profiles, task lists, and comment-based interactions.
• Novelty:
  1. Introduction of shared agent profiles for configuring AI personas.
  2. Explicit task delegation via a shared task list with manual and autonomous triggers.
  3. Integration of AI responses into collaborative comments for familiar and controlled interaction.
• Procedure and key techniques:
  • Users create agents using structured (CV-like) and unstructured (notes) formats.
  • Tasks can be manually triggered or set to execute autonomously based on predefined triggers.
  • AI-generated responses appear as comments, allowing users to accept, reject, or refine suggestions.
  • The system employs a backend with real-time collaboration features and connects to an LLM for AI functionalities.

Results

• Concrete findings:
  • Participants created 39 agents (2.79 per group) and 67 tasks (5.15 per group).
  • AI comments were used as shared action items, with 468 comments logged (376 by AI, 92 by users).
  • Usability ratings: SUS mean score of 66.92 and CSI mean score of 67.21.
  • Final texts averaged 1623 words with a Flesch reading-ease score of 17.71, reflecting academic writing complexity.
• Advantage over baselines:
  • Avoids the need for switching between tools, enabling faster workflows.
  • Supports collaborative creation and use of AI agents tailored to team needs.
  • Provides manual and controlled interaction with AI, addressing trust and verbosity concerns.
• Experiments / evaluation:
  • User study with 30 participants in 14 groups over one week.
  • Mixed methods: interaction logs, semi-structured interviews, and questionnaires.
  • Tasks included real-world writing or predefined essays on AI in daily life.
• Limitations and future work:
  • Limited to academic writers; broader populations need study.
  • Short-term deployment; long-term studies could explore evolving norms.
  • Future comparisons with alternative designs and quantitative studies on agent ownership are suggested.

Summary

This paper introduces a prototype system integrating customizable AI agents into collaborative document editors, addressing the gap in team-based AI writing tools. The study reveals that agent profiles are treated as personal territory, while created agents and their outputs are shared resources. Teams preferred manual control over AI behavior and deliberated on the tradeoff between single and multiple agents for efficiency and perspective value. These findings highlight opportunities and boundaries for treating AI as a shared resource, informing the design of future collaborative writing systems.