Tracing Everyday AI Literacy Discussions at Scale: How Online Creative Communities Make Sense of Generative AI

Paper Title

Tracing Everyday AI Literacy Discussions at Scale: How Online Creative Communities Make Sense of Generative AI

Publication Info

• Topic area: AI literacy in online creative communities
• Keywords: AI literacy, generative AI, Reddit, creative communities, topic modeling, qualitative analysis, temporal analysis, ethics, tool literacy, capacity awareness

Background and Problem

• Problem / challenge: Existing AI literacy frameworks are predominantly expert-driven and top-down, failing to capture how AI literacy organically develops in informal, practice-based settings like online creative communities.
• Significance: Understanding grassroots AI literacy is critical for designing better educational resources, tools, and policies that align with real-world practices and needs.
• Motivation and related work: Prior studies have focused on formal education or expert perspectives, often neglecting informal, community-driven learning environments. Limited research has explored how AI literacy emerges in everyday discussions, especially in creative contexts, and how it evolves over time in response to major AI events.

Solution

• Proposed approach: A mixed-methods analysis of AI literacy discussions in 80 creative-oriented subreddits over three years, using topic modeling, qualitative coding, and temporal analysis.
• Novelty:
  1. A bottom-up, practice-driven account of AI literacy derived from everyday community discussions.
  2. Identification of previously unrecognized literacy practices, such as workflow integration and capability probing.
  3. A longitudinal perspective capturing how AI literacy evolves dynamically in response to major AI events.
  4. Integration of large-scale computational methods with qualitative insights for a nuanced understanding of discourse.
• Procedure and key techniques:
  1. Data collection of 122,506 posts and 1,554,368 comments from Reddit using keyword filtering.
  2. Topic modeling to identify initial themes, followed by qualitative coding of 900 sampled conversations to refine themes.
  3. Classification of all conversations into eight themes, with a focus on four literacy-related themes.
  4. Temporal analysis to track how themes shifted around major AI events.

Results

• Concrete findings:
  • Tool Literacy dominated discussions, comprising 55–60% of posts, with creators focusing on practical skills like setup, troubleshooting, and prompt refinement.
  • Capacity Awareness (15.4%) involved reflections on AI capabilities and limitations, often through probing and testing.
  • Ethics and Responsible Use (11.5%) covered concerns like bias, copyright, and safety.
  • Community Engagement (9.9%) highlighted peer learning through resource sharing, workflow documentation, and feedback.
• Advantage over baselines: The study provides a grassroots, longitudinal view of AI literacy, contrasting with static, expert-driven frameworks. It uncovers practical, socially situated literacy practices not captured in prior work.
• Experiments / evaluation:
  • Dataset: 122,506 posts and 1,554,368 comments from 80 creative subreddits (April 2022–February 2025).
  • Methods: Topic modeling, qualitative coding, and LLM-based classification.
  • Metrics: Accuracy of classification (81%), macro F1-score (77%), and temporal analysis of theme dynamics.
• Limitations and future work:
  • Focused solely on Reddit, which may introduce platform-specific biases.
  • LLM-based classification may misinterpret nuanced language.
  • Future work could expand to other platforms (e.g., Discord, Twitter) and explore professional and educational AI use.

Summary

This study examines how AI literacy emerges and evolves in online creative communities through a large-scale, mixed-methods analysis of Reddit discussions over three years. The findings reveal that AI literacy is predominantly practice-driven, with creators focusing on tool use, troubleshooting, and workflow integration, while ethical and reflective dimensions gain prominence during major AI events. The study challenges traditional, expert-driven literacy frameworks by highlighting the dynamic, socially situated nature of AI literacy. These insights can inform the design of tools, resources, and policies that better align with real-world practices and community needs.