Designing AI Peers for Collaborative Mathematical Problem Solving with Middle School Students: A Participatory Design Study

Paper Title

Designing AI Peers for Collaborative Mathematical Problem Solving with Middle School Students: A Participatory Design Study

Publication Info

• Topic area: AI design for collaborative learning in middle-school mathematics education.
• Keywords: AI peers, collaborative problem solving, middle school mathematics, participatory design, generative AI, scaffolding, group dynamics, student agency, emotional support, multi-agent systems.

Background and Problem

• Problem / challenge: Existing AI systems for education focus primarily on tutor-like roles, offering individualized support rather than facilitating collaborative problem solving (CPS). There is limited understanding of how AI can act as peer collaborators in CPS contexts, especially for middle school students.
• Significance: CPS is a critical pedagogical practice in middle-school mathematics, fostering reasoning and equitable participation. AI peers could address challenges such as stalled group dynamics and limited teacher availability, enhancing both learning and collaboration.
• Motivation and related work: Prior work on AI tutors and collaborative intelligent tutoring systems (ITS) has shown promise but remains rigid, domain-specific, and sensitive to timing. Generative AI tools, such as Large Language Models (LLMs), offer new opportunities but have primarily been explored in individual tutoring contexts. Little research exists on youth-centered perspectives for designing AI peers in CPS.

Solution

• Proposed approach: A participatory design (PD) study with middle school students to co-design AI peers for CPS in mathematics, using a technology probe to elicit feedback and preferences.
• Novelty:
  1. Engaging middle school students in co-designing AI peers for CPS, including replicable camp structures and study instruments.
  2. Empirical insights into students’ social, collaborative, and emotional expectations for AI peers, highlighting tensions between reduced task load and uneven user experience.
  3. Design recommendations for scaffold-first AI peers with adjustable help delivery, assured expertise, and configurable personas.
• Procedure and key techniques:
  • A five-day summer camp with 24 middle school students (grades 6-8), integrating CPS tasks and PD activities.
  • Students experienced human-only CPS and CPS with AI peers via a technology probe, followed by co-design activities to articulate desired AI peer features and behaviors.
  • Data collection included surveys, interviews, and artifacts (e.g., posters, example dialogues).
  • Analysis combined quantitative metrics (e.g., NASA-TLX, GEQ) and qualitative thematic coding.

Results

• Concrete findings:
  • AI peers reduced perceived time pressure (NASA-TLX temporal demand: M = 5.67 vs. 9.62, p = 0.002) but increased operational effort (physical demand: M = 8.48 vs. 4.90, p = 0.014).
  • CPS with AI peers weakened social cohesion (GEQ ATGS: M = 5.21 vs. 6.50, p = 0.029) and task coordination (GEQ GIT: M = 4.76 vs. 5.54, p = 0.047).
  • Students preferred scaffold-first AI peers offering hints, error detection, and visual/tool-based support, with adjustable help levels and a peer-like tone.
• Advantage over baselines: AI peers alleviated time pressure compared to human-only CPS but introduced new interactional frictions, highlighting the need for calibrated scaffolding and improved conversational coherence.
• Experiments / evaluation:
  • NASA-TLX and GEQ surveys measured workload and group dynamics across human-only and AI-assisted CPS conditions.
  • Participatory design artifacts (e.g., posters, dialogues) captured students’ preferences for AI peer features and behaviors.
  • A feature preference survey (20 items) quantified students’ priorities for scaffold-first support and peer tone.
• Limitations and future work:
  • Single-site study during a summer camp limits generalizability; future research should explore regular classroom contexts.
  • No comparison with AI tutors or single-peer AI setups; additional conditions could clarify role-specific preferences.
  • Focused on middle-school mathematics; transferability to other domains and age groups remains uncertain.

Summary

This study explored how AI peers can support collaborative problem solving in middle-school mathematics through participatory design with 24 students. Findings revealed that AI peers reduced time pressure but increased operational effort and strained group dynamics. Students preferred scaffold-first AI peers offering hints, error detection, and visual/tool-based support, with adjustable help levels and a peer-like tone. Recommendations include progressive scaffolding, calibrated personas, and coordinated multi-agent dialogue systems. These insights contribute to designing AI collaborators that preserve student agency, enhance group climate, and support learning in K-12 education.