Mentigo: An Intelligent Agent for Mentoring Students in the Creative Problem Solving Process

Research Background and Issues

What problems or challenges did the authors identify?

• Implementing Creative Problem-Solving (CPS) projects in secondary education requires personalized and continuous guidance. However, due to resource constraints and the diversity of student needs, such guidance is difficult to achieve in large-scale teaching contexts.
• Teachers face challenges in providing adequate support during stages such as team collaboration, task planning, and problem analysis. For secondary school students, the interdisciplinary nature and complexity of tasks are particularly pronounced.
• There is limited research on generative AI in secondary education, as most studies focus on children or adult learners.

Why is this problem important?

• CPS fosters students' creativity and critical thinking, enhancing their ability to solve complex real-world problems, making it a teaching method with significant educational value.
• Promoting CPS in STEM education helps cultivate critical thinking and creativity, which are key skills for secondary school students in their educational and career development.
• There is a gap in the development of generative AI guidance systems tailored to the specific needs of secondary school students.

Research Motivation and Related Work

• Generative Artificial Intelligence (Generative AI) can provide personalized teacher-student interactions for secondary school CPS projects, reducing teachers' workload.
• Previous research has focused on rule-based teaching systems or task assistance models, with limited exploration of generative AI as a mentor role.
• Studies by Chinese scholars and international education experts provide a background, suggesting that generative AI can enhance classroom education, especially in creative tasks.

Solution

What methods or solutions did the authors propose?

• The authors proposed an AI-driven mentor agent system called "Mentigo," specifically designed to guide secondary school students in completing creative problem-solving tasks.
• The Mentigo system uses real classroom interaction data to support a six-stage CPS task process, providing personalized guidance to students through dynamic monitoring and generative AI.

What is innovative about this solution?

• The Mentigo system dynamically updates student state models and provides context-sensitive guidance strategies, shifting from static task guidance systems to real-time responses to students' psychological and cognitive states.
• The system goes beyond basic AI tools for direct task completion by offering long-term, in-depth learning support through dynamic monitoring.
• It is the first to use a real interaction dataset (2,362 interaction records) to build a mentor model that improves creative problem-solving.

What are the implementation steps and key technologies used?

1. Dataset Construction:

   • The authors collected interaction data from 57 secondary school students and 12 mentors during two CPS workshops, totaling 80 hours of classroom recordings, which generated 1,615 dialogue records and 747 guidance strategies.
   • The data were categorized into three key elements: learning stages, student states, and mentor strategies.

2. Architecture Design:

   • Database Module: Stores the dynamic mapping relationships among task stages, students' cognitive and emotional states, and guidance strategies.
   • Controller Agent: Responsible for real-time stage decision-making, state updates, and strategy selection, ensuring a structured task flow.
   • Mentor Agent: Provides personalized feedback and simulates emotionally supportive mentor interactions.

3. Technical Details:

   • The system uses Google Sheets as the backend storage, combined with OpenAI GPT-4 and LangChain technologies for real-time language generation.
   • Retrieval-Augmented Generation (RAG) technology is employed to dynamically search the database for relevant cases, ensuring contextually appropriate guidance.

4. Experiments and Evaluation:

   • User Experiment: Tested Mentigo's performance against a baseline system by comparing students' task completion under both systems, evaluating knowledge gains and learning outcomes.
   • Expert Review: Five education experts reviewed student reports and dialogue data to validate the system's educational value.

Research Outcomes

What specific results were achieved?

• Mentigo demonstrated a significant improvement in student task engagement. Students spent more time on tasks (average 43.7 minutes) and engaged in more dialogue turns (average 24 turns) in the Mentigo environment.
• In terms of knowledge improvement, the post-test scores of the Mentigo group (M=6.67) were significantly higher than those of the baseline system group (M=5.0).
• Learning report evaluations showed that the Mentigo system helped students generate higher-quality, more detailed, and more original task solutions compared to the baseline system.

What advantages does it have over existing solutions?

• Mentigo features dynamic adaptability, real-time monitoring of student states, and adjustment of strategies and task stages, addressing the limitations of traditional static AI teaching tools.
• The mentor agent acts as an "emotional supporter," providing students with a more human-like, interactive experience.

What were the experimental or evaluation results?

• User experiment results indicated that Mentigo outperformed in fostering deep thinking and creativity. Students using the Mentigo system demonstrated higher levels of analytical, evaluative, and creative cognitive language in their dialogues.
• Expert reviews highlighted that Mentigo was more effective than the baseline system in cultivating students' critical thinking and task completion independence.

Limitations and Future Directions

Limitations:

• The current experimental conditions were relatively controlled, with a small scale, and did not fully validate Mentigo's long-term stability in real educational environments.
• The study did not explore cross-disciplinary collaborative tasks or how teachers could intervene in the system's behavior in real-time to support group learning.

Future Directions:

• Apply Mentigo in real classroom settings to explore its performance in different teaching and student collaboration scenarios.
• Develop a teacher interface to enable mentors to monitor student states in real-time and provide support.
• Incorporate multimodal signals (e.g., body language and emotion recognition) to improve the accuracy of the system's student state detection.

Through the development and evaluation of the Mentigo system, this research provides a new direction for the application of AI in education, highlighting the significant potential of generative artificial intelligence in promoting personalized learning.