Testing, Socializing, Exploring: Characterizing Middle Schoolers’ Approaches to and Conceptions of ChatGPT

Title of the Paper

Testing, Socializing, Exploring: Characterizing Middle Schoolers’ Approaches to and Conceptions of ChatGPT

Paper Information

• Subject Area: Human-Computer Interaction and Educational Technology
• Keywords: AI Literacy, Child-AI Interaction, AI Conceptions, Large Language Models (LLMs), Conversational Agents (CAs), ChatGPT, Informal Learning, Generative AI

Research Background and Questions

• What problems or challenges did the authors identify?
  With the rapid integration of generative AI into daily life, adolescents, particularly middle schoolers, are increasingly engaging with these technologies. However, current research on how middle schoolers understand and interact with generative AI is largely confined to formal learning settings like classrooms, with limited exploration of its use in informal learning spaces such as museums.

• Why is this issue important?
  Middle school is a critical period for developing reasoning skills and forming identity. Understanding students’ perspectives and interaction patterns with generative AI can inform the design of more effective AI literacy interventions, helping them develop critical technological perspectives and adapt to future societal changes.

• Research Motivation and Related Work
  Existing studies primarily focus on children’s interactions with intelligent technologies like conversational agents, but there is a lack of research on children’s open-ended interactions with generative AI tools like ChatGPT. Furthermore, current AI education content often emphasizes technical aspects, lacking cultural relevance and informal learning approaches for broader audiences. This paper aims to address these gaps by analyzing middle schoolers’ interactions with ChatGPT and providing insights for designing AI literacy exhibits in museums.

Proposed Solution

• What methods or solutions did the authors propose?
  The authors conducted a focus group study in a museum setting, allowing 24 middle schoolers to interact freely with ChatGPT. They collected and analyzed the prompts, ChatGPT’s responses, and group discussion recordings.

• What is innovative about this solution?

  • Designed an open-ended interaction experience to simulate exploratory interactions in informal learning spaces like museums.
  • Employed a multidimensional qualitative analysis approach (dialogues, prompt types, response content) to comprehensively study the interaction process of middle schoolers.

• What are the implementation steps and key technologies used?

  • Data Collection: Recorded focus group interactions with ChatGPT, including audio, video, and chat logs.
  • Data Analysis: Conducted qualitative analysis to extract participants’ topics of interest, interaction patterns (testing AI, socializing with AI, content exploration), and AI-related concepts and misconceptions. These findings were categorized within an AI literacy framework to evaluate learning opportunities.

Research Outcomes

• What specific outcomes were achieved?

  1. Topics of Interest: Middle schoolers showed diverse interests, including academic topics, hobbies, pop culture, and technology-related themes.
  2. Interaction Patterns: Three interaction patterns were identified:
     • Testing: Focused on assessing ChatGPT’s knowledge and capabilities.
     • Socializing: Treated ChatGPT as a peer for interpersonal-like interactions.
     • Exploring: Focused on the content generated by ChatGPT rather than its entity or attributes.
  3. Concepts and Misconceptions: Various concepts and misconceptions about AI were identified, such as attributing human-like qualities to AI and comparing it to other tools.

• What advantages does it have compared to existing solutions?

  • Provided firsthand interaction data from participants in informal learning spaces.
  • Addressed three critical learning dimensions: AI-generated content, social attributes, and testing capabilities.
  • Proposed specific design considerations to improve museum exhibits and related learning activities.

• What were the experimental or evaluation results?

  • Middle schoolers tended to engage with ChatGPT through topics of personal interest, with some participants demonstrating significant misconceptions about AI mechanisms, particularly regarding data access and operational principles.
  • Socializing groups generated more discussions about AI, providing a conversational foundation for designing learning interventions.

• Limitations and Future Directions

  • Limitations:
    • The study focused on short-term interactions, failing to reveal long-term behavioral changes in AI usage.
    • Participants were limited to a specific geographical area and lacked prior computer science coursework experience.
  • Future Directions:
    • Explore the influence of different variables on interaction patterns across more diverse populations.
    • Develop and test interactive exhibits based on the study’s design insights to enhance learning outcomes.

Discussion and Implications

1. Highlighting Personal and Cultural Relevance: AI education should incorporate diverse topics to align with students’ broad interests, making learning content more engaging.
2. Expanding AI Exploration Scope: Encourage interactions beyond factual inquiries, such as exploring AI ethics and creative capabilities.
3. Leveraging Anthropomorphism to Understand AI: Use anthropomorphism as an accessible way to understand generative AI, while being cautious to avoid inaccurate analogies.
4. Enhancing Creativity-Oriented Learning: Utilize multimodal interactions (e.g., images and voice) to showcase generative AI’s capabilities and support personalized co-creation, fostering deeper learning experiences.

Conclusion

By exploring middle schoolers’ free interactions with generative AI (ChatGPT), this study identified three interaction patterns and their associated learning opportunities. These findings provide valuable insights for designing AI education tools that are inclusive, culturally relevant, and capable of fostering critical reflection.