ClassAid: A Real-time Instructor-AI-Student Orchestration System for Classroom Programming Activities

Paper Title

ClassAid: A Real-time Instructor-AI-Student Orchestration System for Classroom Programming Activities

Publication Info

• Topic area: Real-time AI-driven orchestration systems for programming education.
• Keywords: Generative AI, classroom orchestration, programming education, real-time feedback, TA Agent, instructor dashboard, cognitive assessment, adaptive learning, AI-student interaction, personalized feedback.

Background and Problem

• Problem / challenge: Existing AI tools for programming education lack mechanisms for real-time monitoring, adaptive feedback, and instructor oversight. They are often passive, reactive, and insufficiently aligned with pedagogical goals, leading to risks such as student overreliance, reduced critical thinking, and limited contextual feedback.
• Significance: Addressing these gaps is crucial to improve the quality of programming education, especially in large classrooms where timely, personalized feedback is challenging to deliver manually.
• Motivation and related work: Prior systems like CodeAid and SPHERE provide post-class feedback or asynchronous support but fail to offer real-time, dynamic, and instructor-supervised AI interactions. This paper builds on formative and dynamic assessment theories to design a system that integrates AI-driven feedback with instructor control.

Solution

• Proposed approach: ClassAid, a real-time orchestration system combining an intelligent TA Agent for student support and an instructor dashboard for monitoring and control.
• Novelty:
  1. A six-stage TA Agent framework for personalized, adaptive feedback based on formative and dynamic assessment theories.
  2. Real-time instructor dashboard enabling dynamic adjustment of AI feedback modes and monitoring of student-AI interactions.
  3. Integration of proactive and context-aware feedback to reduce overreliance on AI and foster independent learning.
• Procedure and key techniques:
  1. TA Agent observes student activities, identifies learning obstacles, reviews historical data, and selects appropriate feedback modes (heuristic, technical, auto, or silent).
  2. Feedback is generated based on cognitive level, error type, and learning history, with instructor-controlled adjustments.
  3. Instructor dashboard provides real-time alerts, aggregated class-level analysis, and detailed student performance views for targeted interventions.

Results

• Concrete findings:
  • TA Agent achieved 95.99% accuracy in cognitive-level classification and 94.71% correctness in feedback generation.
  • Students completed tasks with average accuracy scores of 3.87 and 3.97 (out of 5), and the system generated 1,107 feedback messages across two tasks.
  • Instructors made eight class-wide and 22 individual feedback mode adjustments during the session.
• Advantage over baselines:
  • Compared to general AI tools like ChatGPT, ClassAid provided more pedagogically aligned feedback and maintained instructor oversight, enhancing learning outcomes and reducing overreliance.
• Experiments / evaluation:
  • Conducted in a graduate-level data visualization course with 54 students, one instructor, and two TAs.
  • Mixed-method evaluation included quantitative analysis of TA Agent performance, student feedback surveys, and interviews with eight educators.
• Limitations and future work:
  • Scalability to larger classrooms and generalizability to more complex programming tasks remain untested.
  • Future work includes fine-grained feedback strategies, shared control mechanisms between instructors and students, and integration of multi-source learning data.

Summary

ClassAid introduces a novel real-time orchestration system for programming classrooms, integrating an intelligent TA Agent and an instructor dashboard to provide adaptive, personalized feedback while maintaining instructor control. The system demonstrated strong performance in a classroom deployment, with high accuracy in cognitive-level assessment and feedback generation. Students and instructors reported positive experiences, highlighting its pedagogical alignment and usability. Future research will explore scalability, fine-grained feedback, and broader applicability across diverse educational contexts.