Who You Explain To Matters: Learning by Explaining to Conversational Agents with Different Pedagogical Roles

Paper Title

Who You Explain To Matters: Learning by Explaining to Conversational Agents with Different Pedagogical Roles

Publication Info

• Topic area: The impact of conversational agent roles on learning-by-explaining processes.
• Keywords: Conversational agents, learning-by-explaining, pedagogical roles, Tutee, Peer, Challenger, cognitive engagement, metacognition, educational technology.

Background and Problem

• Problem / challenge: Existing research on conversational agents in education focuses on single pedagogical roles, making it unclear how different roles influence learning dynamics, outcomes, and experiences. Prior studies often confound role effects with other factors like task design and content.
• Significance: Understanding how agent roles shape learning processes can guide the design of educational agents tailored to specific pedagogical goals, improving learning outcomes and experiences.
• Motivation and related work: Prior work has explored roles like tutors, peers, and debaters but lacks systematic comparisons of their effects. This study addresses the gap by directly comparing agent roles (Tutee, Peer, Challenger, Control) in a controlled learning-by-explaining context.

Solution

• Proposed approach: A between-subjects study comparing the effects of three conversational agent roles (Tutee, Peer, Challenger) and a control condition on learning dynamics, outcomes, and experiences.
• Novelty:
  1. Empirical evidence on how agent roles shape learning dynamics and experiences.
  2. Theoretically grounded role design based on educational interaction frameworks.
  3. Design implications for tailoring agents to specific pedagogical goals and phases.
• Procedure and key techniques:
  • Participants (N=96) studied an economics concept and explained it to their assigned agent over five interaction rounds.
  • Agents were designed to embody distinct pedagogical roles:
    • Tutee: Simulated a novice learner, prompting participants to teach and clarify concepts.
    • Peer: Acted as a collaborative partner, engaging in co-constructive dialogue.
    • Challenger: Adopted a Socratic questioning style to probe reasoning.
    • Control: Provided minimal interaction, simulating self-explanation.
  • Data collection included pre- and post-tests, behavioral logs, coded interaction patterns, and self-reported surveys.

Results

• Concrete findings:
  • No significant differences in objective learning outcomes (test scores) across conditions.
  • Significant differences in subjective learning experiences:
    • Tutee: Highest cognitive investment but also highest pressure.
    • Peer: Most enjoyable and engaging, with high absorption and low pressure.
    • Challenger: Promoted critical thinking with moderate pressure.
• Advantage over baselines:
  • All three agent roles (Tutee, Peer, Challenger) outperformed the Control condition in perceived critical thinking and competence.
  • Peer and Challenger roles fostered better flow states and lower pressure than Tutee and Control.
• Experiments / evaluation:
  • Participants explained supply-and-demand concepts to agents, applying learned material to a real-world scenario (U.S. Postal Service case).
  • Metrics included interaction patterns (e.g., cognitive/metacognitive acts), word counts, material review frequency/duration, and self-reported measures (e.g., pressure, interest, critical thinking).
• Limitations and future work:
  • Short-term study design limits insights into long-term effects.
  • Single-domain focus (economics) may not generalize to other subjects.
  • Role perception by participants was not formally verified.
  • Future work should explore longitudinal effects, diverse domains, and personalized agent roles.

Summary

This study systematically compared the effects of conversational agent roles (Tutee, Peer, Challenger, Control) on learning-by-explaining processes. While objective test scores did not differ significantly, agent roles shaped interaction patterns and subjective experiences. The Tutee role elicited the most cognitive effort but caused high pressure, the Peer role fostered enjoyable and collaborative learning, and the Challenger role promoted critical thinking with moderate pressure. These findings highlight the importance of tailoring agent roles to specific pedagogical goals and phases, emphasizing the need for adaptive, role-based educational agents. Future research should investigate long-term effects and explore role designs across diverse learning contexts.