An Intelligent Pedagogical Agent to Foster Computational Thinking in Open-Ended Game Design Activities

Document Title

An Intelligent Pedagogical Agent to Foster Computational Thinking in Open-Ended Game Design Activities

Document Information

• Subject Area: Intelligent educational technology and fostering computational thinking in open-ended learning environments
• Keywords: Pedagogical agent, Real-Time Support, Game Design, Computational Thinking, Open-Ended Learning Environments

Research Background and Problem

• Problem or Challenge:

  • Open-ended learning environments (OELEs) can enhance learning outcomes by encouraging students to actively explore, but their relatively loose structure may make it difficult for some students to assess their learning progress or success.
  • The Unity-CT game design learning environment has the potential to inspire students to learn computational thinking (CT), but its complexity means some students require additional support.
  • In remote learning settings, teachers find it challenging to monitor students at all times and provide customized, timely assistance.

• Significance:

  • The U.S. education system is increasingly focused on promoting computational thinking through early education.
  • Intelligent Pedagogical Agents (IPA) can provide personalized and dynamic support in teaching.
  • The widespread adoption of remote learning environments following the COVID-19 pandemic highlights the need to develop educational technologies adapted to this new norm.

• Research Motivation and Related Work:

  • Numerous studies have demonstrated that AI-driven adaptive prompts can enhance learning outcomes, but their application in open-ended game design activities has been limited.
  • Research has primarily focused on programming simulations, while using IPA in free-form game design environments (which are more appealing to younger students) is unprecedented.
  • Existing literature suggests that repetitive interventions can improve learning outcomes, but there is a lack of research on the effects of repeated prompts for younger students.

Solution

• Proposed Solution:

  • Design and evaluate an IPA system to provide targeted operational prompts in the Unity-CT environment, helping students identify and correct common learning errors.
  • Compare different intervention versions: one-time intervention (1-Shot) and repeated intervention (Repeated).

• Innovations:

  • Introduced an AI-based IPA into a commercialized real-world learning system, Unity-CT.
  • Conducted the first study on the effects of repeated interventions for younger students.
  • Addressed the challenge of modeling a large number of unstructured behaviors in open-ended game design tasks.

• Implementation Steps and Key Technologies:

  1. Identify two common errors (3D rotation errors and "Play Mode" editing errors) and develop intervention content for them.
  2. Design user prompts using text bubbles to deliver progressively detailed feedback (problem -> cause -> solution).
  3. Deploy real-time data recording and analysis technology in a remote learning environment.
  4. Compare the behaviors of three student groups:
     • No intervention (No-Intervention)
     • One-time prompt (1-Shot)
     • Repeated prompt (Repeated)
  5. Collect student behavior logs and survey feedback to analyze the immediate effects and long-term impact of the IPA.

Research Outcomes

• Specific Outcomes:

  • The repeated intervention strategy (Repeated) effectively reduced the occurrence of "Play Mode" errors and significantly increased the frequency of correct behaviors.
  • For the 3D rotation problem, although the error rate was visibly high, IPA prompts failed to reduce errors or improve autonomous correction behaviors, possibly due to the operational complexity of the task itself.
  • Students' subjective perceptions were relatively positive, with approximately 82% of students willing to use the IPA again in the future.

• Comparison with Existing Solutions and Advantages:

  • This study is the first to apply IPA to free-form game design CT education tasks and demonstrate its effectiveness in a large-scale commercial remote education environment.
  • Repeated prompts showed clear advantages over one-time prompts in reducing errors and improving behavioral correctness.

• Experimental or Evaluation Results:

  • The 1-Shot intervention did not significantly improve behavior, while "Repeated" reduced "Play Mode" errors and demonstrated significantly higher correctness compared to other groups.
  • Student subjective feedback:
    • Average satisfaction score was approximately 4 (out of 5).
    • The "Repeated" prompt group reported slightly higher levels of "confusion" compared to the 1-Shot group, but overall confusion levels remained low.
    • No students found the IPA overly "intrusive."
  • For "Rotation," the intervention effects were not significant, indicating the need for further optimization of intervention content.

• Limitations and Future Directions:

  • Limitations:
    • The effectiveness of error interventions is highly dependent on task and prompt content design, particularly for complex behavioral operations (e.g., 3D rotation).
    • The sample was limited to a specific environment, which may not be fully generalizable to other educational platforms.
  • Future Directions:
    • Introduce data-driven adaptive interventions for various complex behaviors.
    • Conduct in-depth research on the frequency and methods of repeated prompts for students of different ages.
    • Explore IPA systems centered on student-initiated support requests.