AI as We Describe It: How Large Language Models and Their Applications in Health are Represented Across Channels of Public Discourse

Paper Title

AI as We Describe It: How Large Language Models and Their Applications in Health are Represented Across Channels of Public Discourse

Publication Info

• Topic area: Representation of Large Language Models (LLMs) in public discourse, focusing on health applications.
• Keywords: Large language models, generative AI, public discourse, health applications, risk communication, anthropomorphism, media analysis, TikTok, Reddit, YouTube.

Background and Problem

• Problem / challenge: Public discourse on LLMs in health lacks balanced representation, with insufficient communication of risks and limited explanation of LLMs’ generative nature.
• Significance: Misrepresentation can lead to flawed public understanding, inappropriate mental models, and potential real-world harms in high-stakes domains like health.
• Motivation and related work: Previous studies have shown that media shapes public perceptions of emerging technologies, often emphasizing benefits over risks. However, there is limited research on how LLMs are represented across diverse public discourse channels, particularly in health contexts.

Solution

• Proposed approach: A large-scale, multi-platform analysis of public discourse on LLMs in health, examining lexical style, informational content, and symbolic representation across five channels: news, research press, YouTube, TikTok, and Reddit.
• Novelty:
  1. Comprehensive cross-platform analysis of both professional and layperson-driven content on LLMs in health.
  2. Categorization of six health subdomains for LLM applications to guide future research.
  3. Empirical evidence identifying gaps in literacy, governance, and risk communication.
• Procedure and key techniques:
  • Data collection from five channels (62,783 items reduced to 21,773 after filtering) using health- and LLM-related keywords.
  • Analysis of three dimensions: lexical style (emotional tone, writing style), informational content (framing, risk disclosure, health subdomains), and symbolic representation (anthropomorphism).
  • Statistical tests (Kruskal–Wallis H-tests, Dunn’s post hoc tests) to assess cross-channel differences.

Results

• Concrete findings:
  • Public discourse on LLMs in health is generally positive and increasingly so over time, with TikTok and YouTube showing the most optimistic tone.
  • Risk communication is limited, with most risks framed as information quality issues (e.g., hallucinations, inaccuracies) and little mention of LLMs’ generative nature.
  • Health subdomains emphasized vary by platform: TikTok and Reddit focus on consumer health (e.g., mental health support), while news and research press highlight clinical and research applications.
  • Anthropomorphism is generally low but more variable on TikTok and Reddit.
• Advantage over baselines: Provides the first large-scale, multi-platform analysis of LLM discourse in health, bridging gaps in understanding cross-channel differences and their implications for public perception.
• Experiments / evaluation:
  • Data from December 2022 to December 2024.
  • Metrics include emotional tone, framing types, risk disclosure frequency, and anthropomorphism scores.
  • Validation of annotations using human and cross-model agreement (F1-macro scores: 0.74–0.85).
• Limitations and future work:
  • Focus on English-language content and U.S.-centric platforms may limit generalizability.
  • Platform-specific biases (e.g., TikTok API cutoff, Pushshift limitations) and reliance on a single annotation model.
  • Future work should explore multilingual, cross-cultural contexts and more fine-grained aspects of discourse.

Summary

This study analyzes how public discourse introduces LLMs in health across five channels, revealing generally positive narratives with limited risk communication and rare explanations of LLMs’ generative nature. Professional outlets focus on clinical and systemic implications, while layperson platforms emphasize consumer health, particularly mental health support, with greater variation in tone and anthropomorphism. These findings highlight gaps in literacy and governance, suggesting the need for improved communication strategies and regulatory efforts to address high-demand but high-risk domains. The study underscores the role of public discourse as a diagnostic tool for identifying public attention, literacy gaps, and governance needs.