Human-Human-AI Triadic Programming: Uncovering the Role of AI Agent and the Value of Human Partner in Collaborative Learning

Paper Title

Human-Human-AI Triadic Programming: Uncovering the Role of AI Agent and the Value of Human Partner in Collaborative Learning

Publication Info

• Topic area: Collaborative programming with AI in educational contexts
• Keywords: Human-AI collaboration, triadic programming, collaborative learning, AI-assisted programming, social presence, shared AI, personal AI, proactive AI, programming education, accountability

Background and Problem

• Problem / challenge: Existing research on AI-assisted programming often focuses on human-AI dyads, positioning AI as a replacement for human collaboration. This overlooks the social and pedagogical benefits of human-human collaboration, such as shared reasoning and accountability.
• Significance: Collaborative programming fosters critical thinking, problem-solving, and social engagement, which are essential for learning and professional development. Understanding how AI can augment rather than replace human collaboration is crucial for designing effective educational tools.
• Motivation and related work: Prior studies have shown that AI tools like GitHub Copilot and ChatGPT improve productivity and code quality but lack the social presence and mutual engagement of human-human collaboration. Research on multi-human-AI collaboration is limited, particularly in programming contexts, leaving open questions about how AI can be integrated to enhance learning and teamwork.

Solution

• Proposed approach: Human-Human-AI (HHAI) triadic programming, where two humans collaborate with an AI agent as a third teammate, either in a shared or personal AI configuration.
• Novelty:
  1. Empirical evidence comparing human-human-AI triadic programming with human-AI dyads, highlighting enhanced collaborative learning and social presence.
  2. Conceptual framing of AI as an augmentation tool rather than a replacement in collaborative programming.
  3. Design principles for AI-supported collaborative programming, emphasizing accountability, conversational flow, and reinforcement of pedagogical benefits.
• Procedure and key techniques:
  1. Developed an integrated system with a collaborative code editor, conversational interface, and proactive AI interventions.
  2. Conducted a within-subjects study with 20 participants (10 pairs) across three conditions: Shared AI, Personal AI, and Human-AI baseline.
  3. Analyzed collaborative learning, social presence, AI usage, and programming performance using questionnaires, qualitative interviews, and interaction data.

Results

• Concrete findings:
  • Both HHAI conditions (Shared and Personal AI) significantly improved collaborative learning (CLS: Shared AI β = 0.93, p < .001; Personal AI β = 0.55, p = .028) and social presence (SPQ: Shared AI β = 1.43, p < .001; Personal AI β = 1.04, p = .007) compared to the HAI baseline.
  • Shared AI heightened participants' sense of responsibility for understanding AI suggestions (M = 6.05, p < .001) and reduced reliance on AI-generated code (Shared AI: 1.4%, HAI: 23.1%).
  • Proactive AI interventions were less disruptive in Shared AI (M = 2.71, p < .001) compared to HAI (M = 6.01) and Personal AI (M = 6.18).
  • Programming performance (subtasks completed) was consistent across conditions.
• Advantage over baselines:
  • HHAI conditions restored social and pedagogical benefits absent in HAI, such as peer accountability, dialogue, and reduced overreliance on AI.
  • Shared AI better aligned with group flow and reduced disruptiveness compared to Personal AI.
• Experiments / evaluation:
  • Participants: 20 computer science students (10 pairs) with prior programming and AI experience.
  • Tasks: Three LeetCode-style problems with multiple subtasks.
  • Metrics: Collaborative Learning Scale, Social Presence Questionnaire, AI usage, and programming performance.
• Limitations and future work:
  • Lack of a human-human baseline limits comparison of AI's added value.
  • Study focused on students from a single institution and short-term tasks, limiting generalizability.
  • Familiarity between participants may have influenced collaboration dynamics.
  • Future work: Longitudinal studies, diverse populations, and scaffolding for effective human-to-human interaction.

Summary

This study introduces human-human-AI triadic programming, where an AI agent augments human collaboration rather than replacing it. Through a within-subjects study, the authors demonstrate that HHAI enhances collaborative learning, social presence, and responsible AI use compared to human-AI dyads. Shared AI configurations were particularly effective in fostering accountability and aligning with group flow. While programming performance remained consistent across conditions, HHAI encouraged richer conversational behaviors and reduced reliance on AI-generated code. These findings suggest that embedding AI within peer collaboration can strengthen the social and pedagogical dynamics of learning, offering design principles for future AI-supported educational tools.