A Piece of Theatre: Investigating How Teachers Design LLM Chatbots to Assist Adolescent Cyberbullying Education

Document Title

A Piece of Theatre: Investigating How Teachers Design LLM Chatbots to Assist Adolescent Cyberbullying Education

Document Information

• Subject Area: Educational Technology, Intersection of Artificial Intelligence and Social Issues
• Keywords: Large Language Models (LLM), Chatbots, Cyberbullying, Education, Teachers, Role-Playing, Collaborative Design, Social-Emotional Learning

Research Background and Issues

• Identified Problems or Challenges:

  1. Cyberbullying negatively impacts adolescents' mental health. Teachers need to help students develop "bystander intervention" skills, but educational resources are limited.
  2. Expanding personalized teaching using chatbots (e.g., based on large language models) presents challenges in both technology and content design.
  3. Teachers are no longer mere knowledge transmitters but act as educational designers and behavior moderators. However, they lack tools to understand, design, and control LLM-based chatbots.

• Significance: Cyberbullying is widespread, with consequences including depression, self-harm, and even suicide. Training students to become "upstanding bystanders" can effectively alleviate victims' suffering and foster a healthier online environment.

• Research Motivation and Related Work:

  1. Previous studies have shown that chatbots can provide personalized, interactive guidance, but most remain at the prototype stage in laboratories, lacking feasibility assessments for classroom application.
  2. There is a lack of tools enabling teachers to customize chatbot designs, preventing them from meeting classroom teaching goals and addressing students' contextual needs.
  3. Research needs to explore how teachers envision and use these tools and how they integrate them into existing curricula.

Solution

• Proposed Method or Solution:

  1. Developed a no-code chatbot design tool named Co-Pilot, implemented with LLM-Chains, allowing teachers to create complex dialogue flows without programming.
  2. The tool consists of two components: Chatbot Builder and Chatbot Tester. The Builder enables teachers to create dialogue logic through drag-and-drop functionality, while the Tester simulates real-world online scenarios to help teachers evaluate chatbot effectiveness.
  3. Conducted user research involving 13 secondary school teachers to gather their design requirements and feedback.

• Innovations:

  1. Teachers are empowered as "playwrights," designing interactive scripts between students and the chatbot, while allowing the chatbot to improvise within a defined framework.
  2. Integrated functionality to support students in repeated role-playing practice, enabling them to explore potential behavioral responses in a safe environment.
  3. Provided a method to combine controlled dialogues from LLMs with classroom-specific educational objectives.

• Key Technologies and Implementation Steps:

  • Designed a tree-like dialogue flow structure where teachers define student behavior nodes and configure chatbot response functions.
  • Applied the visual programming concept of PromptChainer for LLMs to reduce technical complexity while offering fine-grained dialogue control.
  • Incorporated student simulations to generate virtual student responses, challenging teachers' assumptions about student behavior.

Research Outcomes

• Specific Results:

  1. Teachers widely praised the Co-Pilot tool, finding its design intuitive and capable of achieving educational goals.
  2. The study revealed that teachers tend to view chatbots as part of an "educational theatre," using scripts to guide student role-playing and foster social-emotional skills.
  3. Teachers were enthusiastic about personalizing chatbots to reflect classroom-specific culture, language, and teaching styles.

• Advantages Compared to Existing Solutions:

  1. Co-Pilot enables higher levels of behavioral guidance and interaction, allowing chatbots to "improvise" without deviating from the teacher's predefined direction.
  2. The design tool enhances students' interactive experiences, enabling repeated practice and strategy adjustments to better meet their actual learning needs.

• Experimental or Evaluation Results:

  1. Tests showed that teachers could quickly familiarize themselves with and apply the Co-Pilot tool to create chatbots reflecting their teaching preferences.
  2. When students used these chatbots, they were able to experiment with different behaviors, cultivating empathy and situational awareness.

• Limitations and Future Directions:

  1. Teachers still face challenges in simulating student behavior. Future developments should include tools to assist teachers in script generation, leveraging LLMs for automated suggestions.
  2. Current dialogue structures focus on single-user interactions. Future research should support multi-participant role-playing and explore how to manage more complex social interactions.
  3. Further studies are needed to address potential biases in LLMs and mitigate risks in sensitive educational scenarios.

By employing the metaphor of teachers as "playwrights," this research successfully highlights the potential of LLM-based chatbots to support cyber safety and social-emotional education, while also identifying current design tool limitations and directions for future improvement.