When AI Rewrites the News: How Sentiment, Framing, and LLM Disclosure Shape Perceptions

Paper Title

When AI Rewrites the News: How Sentiment, Framing, and LLM Disclosure Shape Perceptions

Publication Info

• Topic area: The impact of AI-modified news on reader perceptions of bias, trust, and emotional engagement.
• Keywords: AI-mediated journalism, sentiment analysis, framing effects, LLM disclosure, trust in news, emotional engagement, political communication, media bias, transparency, HCI.

Background and Problem

• Problem / challenge: Limited understanding of how AI-modified news articles, particularly in terms of sentiment, framing, and disclosure, affect reader perceptions of bias, trust, and emotional responses. Prior studies often examine these factors in isolation, leaving gaps in understanding their combined effects.
• Significance: With the increasing use of LLMs in journalism, understanding these dynamics is critical to addressing concerns about polarization, trust erosion, and the ethical use of AI in news production.
• Motivation and related work: Previous research highlights the influence of sentiment and framing on audience perceptions and the challenges of transparency in AI-generated content. However, the interactive effects of these variables, particularly in politically charged contexts, remain underexplored. This study builds on these gaps by experimentally manipulating sentiment, framing, and disclosure in AI-modified news articles.

Solution

• Proposed approach: A 2×2 between-subjects experiment to analyze the effects of sentiment (neutral vs. extreme), framing (balanced vs. one-sided), and LLM disclosure on reader perceptions of bias, trust, and emotional engagement.
• Novelty:
  1. Empirical evidence on how sentiment and framing jointly influence perceptions of bias and trust in political news.
  2. A methodological framework for generating and validating controlled sentiment-framing manipulations using multiple LLMs.
  3. Design implications for AI-assisted news systems, emphasizing transparency, framing awareness, and sentiment calibration.
• Procedure and key techniques:
  • News articles on contentious topics were rewritten using LLMs (GPT, Grok, Gemini, Claude) into four conditions: Neutral–Balanced, Extreme–Balanced, Neutral–One-Sided, and Extreme–One-Sided.
  • Articles were validated by human reviewers and computational tools for sentiment and framing accuracy.
  • 225 U.S.-based participants evaluated articles for bias, trustworthiness, emotional response, and agreement, both pre- and post-disclosure of LLM involvement.
  • Data were analyzed using ANOVAs and thematic analysis of open-ended responses.

Results

• Concrete findings:
  • Extreme sentiment increased perceived bias and amplified negative emotions (e.g., anger, disgust, anxiety), while happiness remained stable.
  • Balanced framing with extreme sentiment heightened surprise and suspicion, consistent with the Hostile Media Effect.
  • Disclosure of LLM involvement slightly reduced trustworthiness, particularly for extreme or one-sided articles, but had minimal effect on perceived bias or balance.
• Advantage over baselines:
  • Neutral sentiment and balanced framing were generally rated as more credible, though balanced framing sometimes paradoxically increased perceptions of manipulation.
  • Extreme sentiment consistently worsened emotional and trust outcomes compared to neutral sentiment.
• Experiments / evaluation:
  • Participants (N=225) read AI-modified articles and rated them on bias, trustworthiness, emotional response, and balance.
  • Emotional reactions (e.g., anger, anxiety) were stronger for extreme sentiment conditions, while trust declined slightly post-disclosure.
  • Articles were validated using human coders and computational metrics (e.g., sentiment polarity, framing entropy).
• Limitations and future work:
  • Single-article design limits generalizability; future studies should include multiple topics.
  • MTurk sample may not fully represent broader populations; cross-national studies are needed.
  • Effects of varying disclosure formats and longitudinal impacts remain unexplored.

Summary

This study investigates how sentiment, framing, and LLM disclosure in AI-modified news articles shape reader perceptions of bias, trust, and emotional engagement. Extreme sentiment amplified negative emotions and perceived bias, while balanced framing sometimes increased suspicion when paired with extreme tone. Disclosure of AI involvement slightly reduced trust, particularly for extreme or one-sided articles, but had minimal effect on perceived balance. The findings highlight the need for transparency, emotion-aware moderation, and editorial oversight in AI-mediated journalism to mitigate polarization and maintain trust. Future work should explore more diverse topics, populations, and disclosure strategies to enhance the design of AI-assisted news systems.