Exploring The Impact of Proactive Generative AI Agent Roles In Time-Sensitive Collaborative Problem-Solving Tasks

Paper Title

Exploring The Impact of Proactive Generative AI Agent Roles In Time-Sensitive Collaborative Problem-Solving Tasks

Publication Info

• Topic area: Human-AI collaboration in co-located, time-sensitive problem-solving tasks.
• Keywords: Generative AI, proactive agents, facilitator role, peer role, escape rooms, group collaboration, time-sensitive tasks, workload, coordination, human-AI teams.

Background and Problem

• Problem / challenge: Limited understanding of how proactive generative AI agents influence real-time, co-located teamwork, particularly in high-pressure collaborative environments. Questions remain about whether AI should act as a peer or facilitator and how these roles affect group performance and processes.
• Significance: Insights into AI roles could enhance collaborative problem-solving in domains such as emergency response, healthcare, and cybersecurity, where effective teamwork under time constraints is critical.
• Motivation and related work: Prior research highlights the promise of generative AI and proactive systems in augmenting teamwork, but most studies focus on one-to-one interactions or speculative designs. Human-AI teams (HATs) emphasize interdependence, yet empirical studies on proactive AI roles in group contexts are scarce.

Solution

• Proposed approach: Development and evaluation of two generative AI agents—Fiona (facilitator) and Ava (peer)—embedded in co-located escape-room tasks to study their impact on group performance and processes.
• Novelty:
  1. Functional technology probes of proactive AI agents in facilitator and peer roles.
  2. Empirical study of proactive AI roles in co-located, time-sensitive collaborative problem-solving tasks.
  3. Design considerations for integrating proactive AI agents into teamwork.
• Procedure and key techniques:
  • Tasks: Escape-room puzzles requiring active communication and coordination among four participants.
  • AI agents: Fiona provided summaries and coordination cues; Ava contributed ideas and memory support.
  • Study design: Within-subjects study with 24 participants (6 groups), counterbalanced across three conditions (no AI, peer AI, facilitator AI).
  • Data collection: Surveys (NASA-TLX, AI Perception, Perceived Coordination), performance scores, and focus-group interviews.

Results

• Concrete findings:
  • Performance: Facilitator condition yielded the highest scores (EMM = 14.50), followed by no AI (EMM = 9.33), and peer condition (EMM = 4.67).
  • Workload: Peer agent increased workload significantly compared to facilitator and no AI conditions.
  • AI perception: Peer agent rated higher for improving team coordination but introduced cognitive burden and disrupted flow.
• Advantage over baselines:
  • Facilitator agent supported coordination indirectly, leading to higher scores despite being perceived as less impactful.
  • Peer agent provided timely hints and memory support but disrupted group dynamics and increased effort.
• Experiments / evaluation:
  • Mixed-methods study with quantitative (ART-ANOVA) and qualitative (thematic analysis) approaches.
  • Measures: Task performance, perceived coordination, workload, and AI perception.
  • Context: Escape-room puzzles with varying difficulty levels.
• Limitations and future work:
  • Sample: Participants were pre-established teams, limiting generalizability to ad hoc groups.
  • Task design: Escape-room puzzles may not fully reflect real-world collaborative complexity.
  • Fixed roles: Agents did not adapt to group progress or needs; future work could explore adaptive and hybrid roles.

Summary

This study investigates the roles of proactive generative AI agents in co-located, time-sensitive collaborative problem-solving tasks. The facilitator agent improved group performance indirectly through coordination cues, while the peer agent provided timely hints but increased workload and disrupted flow. Participants perceived the peer agent as more impactful despite its lower performance outcomes. Findings highlight the importance of timing, relevance, and adaptability in AI contributions. Future work should explore adaptive mechanisms and test these roles in more complex, real-world settings.