"Don't Forget the Teachers": Towards an Educator-Centered Understanding of Harms from Large Language Models in Education

Research Background and Issues

• What problems or challenges did the authors identify?

  • Large-scale language models (LLMs) are rapidly entering educational technology (EdTech), providing support for teaching and learning processes. However, the potential negative impacts of these technologies have not been thoroughly studied, especially in the unique domain of education.
  • The specific characteristics of educational environments include: students are often children, making them particularly susceptible to technological influences; learning objectives go beyond obtaining correct answers to understanding problem-solving processes; and fostering high-level skills such as critical thinking and collaboration across various contexts.
  • There is a disparity between EdTech providers and educators in understanding and addressing the risks posed by LLMs. EdTech providers primarily focus on technical hazards, whereas educators are more concerned with the broader impacts on interactions among students, teachers, and school systems.

• Why is this issue important?

  • LLM technologies will profoundly influence the future of education, closely tied to critical aspects such as children's learning dynamics, privacy protection, social development, and educational equity.
  • Existing risk classifications for LLMs are typically domain-agnostic and cannot be directly applied to education, necessitating the development of a new framework tailored to the specific conditions of education.
  • Education is a complex process that cannot be automated or directly observed, requiring more cautious research and implementation.

• Research Motivation and Related Work

  • The authors referenced existing LLM risk classifications, as well as ethical analyses and practical studies on artificial intelligence in education.
  • They proposed a method to incorporate educators' perspectives into the technology design process, aiming to bridge the cognitive gap between EdTech providers and educators for more effective risk mitigation.

Solutions

• What methods or solutions did the authors propose?

  • Through semi-structured interviews with EdTech providers (6 participants) and educators (23 participants), the authors developed a framework for reviewing potential harms of LLMs specific to the education domain.
  • Key methods include:
    1. Systematically analyzing how EdTech providers and educators perceive, evaluate, and mitigate potential harms.
    2. Highlighting differences in their understanding of risks.
    3. Recommending the design and development of EdTech tools centered around educators' needs.

• What is innovative about this solution?

  • The authors combined existing domain-agnostic LLM risk classifications with education-specific harms (e.g., impacts on student learning and social development, teacher workload, and educational equity) to create an education-specific framework.
  • They introduced the perspective of educators to optimize EdTech tool design, emphasizing the intermediary role of educators and participatory design.
  • They proposed interdisciplinary collaboration involving EdTech providers, researchers, regulators, and school leaders in tool design and policy formulation.

• What are the implementation steps and key technologies used?

  • Implementation steps include:
    1. Conducting semi-structured interviews to gather feedback on the real-world use and potential harms of LLMs.
    2. Performing thematic analysis of interview content to identify core issues and mitigation strategies.
    3. Comparing the risk concerns and mitigation measures of EdTech providers and educators.
    4. Proposing policy and design recommendations, such as promoting educator involvement in tool design and establishing clear regulatory review mechanisms.
  • Key technologies include LLM-based toxicity detection, privacy protection mechanisms, retrieval-augmented generation (RAG) storage, and educators' instructional intervention strategies.

Research Outcomes

• What specific outcomes were achieved?

  • The authors identified three major categories of LLM risks in education:
    1. Technical harms: Including harmful content generation, privacy violations, and hallucinated outputs.
    2. Human-machine interaction harms: Such as academic dishonesty.
    3. Broad impact harms: Including suppression of student learning and social development, increased teacher workload, reduced teacher autonomy, and exacerbated educational inequality.
  • They proposed an education-specific framework to complement existing domain-agnostic LLM risk classifications.

• How does this solution compare to existing ones?

  • By focusing on educators' feedback, the framework covers a broader range of harms highly relevant to educational practice.
  • It positions educators as central to EdTech design and risk mitigation, enhancing the user-centeredness of tool design.
  • The interdisciplinary collaboration recommendations promote educational equity and responsible technology use.

• What were the experimental or evaluation results?

  • Interviews revealed significant differences in risk concerns between EdTech providers and educators. Providers focused more on technical-level detection and mitigation, while educators emphasized the long-term impacts of tools on learning quality, social interactions, and teacher autonomy.
  • The study found that educators' proactive interventions in teaching could effectively mitigate some technical harms, but broader systemic social harms require external regulation and collaboration to address.

• Limitations and Future Directions

  • Limitations:
    • Data sources were primarily from English-speaking countries (e.g., the United States, United Kingdom, Canada), excluding perspectives from global education, particularly non-English-speaking and non-Western countries.
    • The sample size of EdTech providers was relatively small, limiting comprehensive market coverage.
    • The study did not directly evaluate the effectiveness of actual tools but relied on interviews to establish a theoretical framework.
  • Future Directions:
    • Expand the research scope to include educators from more countries and regions.
    • Conduct experimental studies in real educational settings to validate the framework's applicability.
    • Explore teacher-led customization of LLM tools to better support diverse teaching needs.

Through this study, the authors not only developed an education-specific LLM risk framework but also proposed strategies for design and mitigation centered on educators, promoting the responsible use of educational technology.