Exploring the Design and Impact of Interactive Worked Examples for Learners with Varying Prior Knowledge

Paper Title

Exploring the Design and Impact of Interactive Worked Examples for Learners with Varying Prior Knowledge

Publication Info

• Topic area: Intelligent tutoring systems and adaptive learning interventions
• Keywords: Intelligent tutoring systems, worked examples, ICAP framework, propositional logic, prior knowledge, cognitive engagement, debugging, adaptive learning, problem solving, behavior analysis

Background and Problem

• Problem / challenge: Traditional worked examples often fail to cater to learners with varying prior knowledge, with low prior knowledge learners benefiting more than high prior knowledge learners. Designing interventions that balance cognitive engagement and scaffolding for diverse learners remains a challenge.
• Significance: Addressing this issue can improve learning outcomes for a broader range of students, particularly in complex domains like propositional logic, where learners' prior knowledge varies significantly.
• Motivation and related work: Prior research highlights the limitations of passive worked examples and the expertise reversal effect, where high prior knowledge learners may not benefit from detailed guidance. The ICAP framework suggests that higher cognitive engagement leads to better learning outcomes, but empirical evidence on designing interactive worked examples for varied cognitive engagement levels is limited. This paper builds on these gaps by introducing and evaluating two novel interactive worked examples.

Solution

• Proposed approach: Development and evaluation of two new types of worked examples—Buggy examples (students fix errors in solutions) and Guided examples (students reconstruct missing parts of solutions)—within an intelligent logic tutoring system.
• Novelty:
  1. Design of two interactive worked example types (Buggy and Guided) aligned with the ICAP framework to promote higher cognitive engagement.
  2. Empirical evidence of differential effectiveness of these interventions based on learners' prior knowledge.
  3. Application of Markov models to analyze and visualize problem-solving behavior patterns.
  4. Design implications for adaptive and personalized scaffolding in intelligent tutoring systems.
• Procedure and key techniques:
  • Buggy examples present solutions with intentional errors for students to identify and correct, requiring debugging and critical thinking.
  • Guided examples provide partial solutions with step-specific hints, focusing on active reconstruction of proofs.
  • A controlled experiment with 155 undergraduate students in a logic tutor compared the effects of Buggy, Guided, and traditional worked examples on learning outcomes, using pretest-posttest designs and behavior analysis through interaction logs.

Results

• Concrete findings:
  • Both Buggy and Guided groups outperformed the Control group in posttest scores (Buggy: 72.3, Guided: 72.4, Control: 67.8).
  • Guided examples significantly improved rule application accuracy for low prior knowledge learners (78.7% vs. 71.8% in Control).
  • Buggy examples significantly improved rule application accuracy for high prior knowledge learners (82.6% vs. 74.4% in Control).
  • Both interventions reduced problem completion time compared to Control (Buggy: 8.4 minutes, Guided: 8.9 minutes, Control: 11.7 minutes).
• Advantage over baselines:
  • Buggy examples benefited high prior knowledge learners through productive struggle and debugging practice.
  • Guided examples supported low prior knowledge learners by reducing cognitive load and providing structured scaffolding.
• Experiments / evaluation:
  • Participants: 155 undergraduate Computer Science students.
  • Conditions: Control (traditional worked examples), Buggy, and Guided groups.
  • Metrics: Problem scores, rule application accuracy, problem completion time, and solution length.
  • Analysis: Mixed-effects regression and Markov models for behavior analysis.
• Limitations and future work:
  • Lack of qualitative data on learners' affective states and study environments.
  • Absence of hints in Buggy examples may have increased unproductive struggle.
  • Static assignment of interventions based on pretest scores; future work should explore adaptive problem selection.
  • Limited generalizability to less structured domains or tasks with different complexity levels.

Summary

This study introduces two novel interactive worked examples—Buggy and Guided—within an intelligent logic tutor, designed to cater to learners with varying prior knowledge. Buggy examples benefited high prior knowledge learners by fostering critical thinking through debugging, while Guided examples supported low prior knowledge learners by reducing cognitive load with structured scaffolding. Both interventions led to improved posttest performance and reduced problem completion time compared to traditional worked examples. The study highlights the importance of aligning instructional design with learners' cognitive engagement levels and prior knowledge, offering actionable insights for adaptive learning systems in complex problem-solving domains. Future work should focus on integrating adaptive scaffolding and exploring broader applications across domains.