Exploring Teacher-Chatbot Interaction and Affect in Block-Based Programming

Paper Title

Exploring Teacher-Chatbot Interaction and Affect in Block-Based Programming

Publication Info

• Topic area: Teacher interaction with AI chatbots in educational programming environments.
• Keywords: AI chatbots, block-based programming, computational thinking, teacher perceptions, scaffolding, middle school education, LLMs, usability, differentiation, pedagogical practices.

Background and Problem

• Problem / challenge: Teachers face challenges integrating AI tools like chatbots into classrooms due to concerns about over-reliance, diminished critical thinking, and usability issues. Existing tools often lack features tailored to diverse learners and pedagogical needs.
• Significance: Understanding teacher perspectives is crucial for designing AI tools that enhance learning without undermining foundational skills or teacher autonomy.
• Motivation and related work: Prior studies have explored teacher–AI interactions, emphasizing cognitive, socio-emotional, and artifact-mediated dimensions. However, detailed evidence on teachers’ experiences with LLMs in block-based programming remains limited, particularly regarding their affective responses and preferred design features.

Solution

• Proposed approach: A study involving middle school science teachers interacting with an LLM-powered chatbot integrated into a block-based programming environment (Stax.fun).
• Novelty:
  1. Identification of three teacher personas (Explorer/Task-Focused, Mixed, Frustrated) based on emotional and behavioral profiles.
  2. Analysis of affective responses across activities, highlighting the interplay between system design and teacher emotions.
  3. Integration of TAM and TPB frameworks to model teachers’ intentions to adopt chatbots.
  4. Design recommendations for scaffolding chatbot use, differentiation, and accessibility.
• Procedure and key techniques:
  • Teachers engaged with the chatbot across 11 structured activities, including coding tasks and wave simulations.
  • Data collection involved Zoom recordings, think-aloud transcripts, and interaction traces.
  • Multi-source analysis segmented tasks into timestamps and coded emotional responses.
  • Thematic analysis of post-activity discussions identified benefits, risks, and pedagogical strategies.

Results

• Concrete findings:
  • Positive emotions (e.g., excitement, curiosity) were most frequent during exploratory tasks and initial chatbot interactions.
  • Negative emotions (e.g., frustration, confusion) arose from interface issues and unclear system expectations.
  • Teachers valued the chatbot for boosting student self-efficacy, prompting skills, and differentiation but expressed concerns about over-reliance and diminished critical thinking.
• Advantage over baselines: The study provides unique insights into teacher personas and their scaffolding needs, which are not addressed by existing LLM-integrated IDEs.
• Experiments / evaluation:
  • Participants: 25 middle school teachers (8 lead, 17 novice).
  • Environment: Stax.fun platform with four chatbot modes (Code, Q&A, Coach, Prompts).
  • Metrics: Emotional valence, task completion, help-seeking behavior, thematic analysis of discussions.
• Limitations and future work:
  • Limited scope to actual classroom implementation; findings are based on professional development workshops.
  • Future work should explore long-term impacts of chatbot use on student learning and teacher adoption.

Summary

This study examined middle school teachers’ interactions with an LLM-powered chatbot in a block-based programming environment, identifying three personas with distinct emotional and behavioral profiles. Teachers appreciated the chatbot’s ability to support differentiation, boost student confidence, and reduce their workload, but raised concerns about over-reliance and diminished critical thinking. The integration of TAM and TPB frameworks provided a conceptual model for understanding teachers’ intentions to adopt AI tools. Design recommendations include scaffolding chatbot use, enabling teacher control over features, and adapting the chatbot for diverse learners. These findings contribute to the development of AI tools that align with pedagogical goals and enhance learning outcomes.