Narratives and Perspectives: How AI Summaries Steer Users' Opinions and Engagement on Social Media

Paper Title

Narratives and Perspectives: How AI Summaries Steer Users' Opinions and Engagement on Social Media

Publication Info

• Topic area: Influence of AI-generated summaries on user opinions and engagement in social media contexts.
• Keywords: AI summaries, social conformity, polarisation, social media, user engagement, opinion dynamics, toxic discourse, civil discourse, HCI, AI ethics.

Background and Problem

• Problem / challenge: The influence of AI-generated summaries on user opinions and engagement remains underexplored, particularly regarding their potential to reinforce social conformity, distort perceptions of balance, and exacerbate polarisation.
• Significance: Understanding these effects is critical as AI tools are increasingly integrated into social media platforms, shaping public discourse and potentially impacting democratic processes.
• Motivation and related work: Prior studies have examined AI's role in mitigating misinformation and polarisation but have not fully addressed the psychological and behavioural impacts of non-interactive AI tools like comment summaries. This paper fills the gap by investigating how AI-generated summaries of narratives and agreement percentages influence user opinions and engagement.

Solution

• Proposed approach: The study evaluates two types of AI-generated summaries—textual narrative summaries and percentage breakdowns of agreement/disagreement—through a controlled experiment on simulated Reddit threads.
• Novelty:
  1. Demonstrates how AI percentages amplify social conformity toward majority views.
  2. Shows that AI narrative summaries create misperceptions of balance, reducing opinion shifts.
  3. Highlights the role of thread toxicity in deterring user engagement, independent of AI tools.
  4. Provides design recommendations to mitigate polarising effects of AI tools.
• Procedure and key techniques:
  • Conducted a 144-participant experiment with four conditions: control (no AI tools), AI percentages only, AI narratives only, and both tools combined.
  • Participants read six simulated Reddit threads (three civil, three toxic) with varying polarisation levels (balanced, polarised-agree, polarised-disagree).
  • Measured opinion change (magnitude and direction), perceived polarisation, and willingness to engage using Likert scales and qualitative feedback.

Results

• Concrete findings:
  • AI percentages significantly increased opinion shifts toward majority views (Cohen’s d = 0.81; up to 1.5 Likert swing).
  • AI narratives reduced the magnitude of opinion change (0.9 Likert swing) by creating a false perception of balance.
  • Combined tools produced the largest opinion shifts (1.5 Likert swing).
  • No significant effect of AI tools on willingness to engage; thread civility was the primary driver of engagement.
• Advantage over baselines:
  • AI percentages amplified conformity more than reading comments alone.
  • AI narratives moderated opinion shifts compared to the control condition.
• Experiments / evaluation:
  • Mixed-methods approach: Generalised Linear Mixed Models (GLMMs) and qualitative thematic analysis.
  • Simulated Reddit threads with controlled civil and toxic discourse.
  • Topics included Fukushima wastewater, public transport, social media under-16 ban (civil), Ukraine war, gun control, and Elon Musk’s government role (toxic).
• Limitations and future work:
  • Simulated environments may not fully capture real-world dynamics.
  • Future studies should explore nuanced agreement levels, platform-specific effects, and the impact of misinformation in AI summaries.

Summary

This study reveals that AI-generated summaries significantly influence user opinions and perceptions in social media threads. AI percentages amplify conformity to majority views, while AI narratives reduce opinion shifts by creating a false sense of balance. Thread civility, rather than AI tools, drives user engagement. These findings underscore the need for responsible AI design to mitigate polarisation and promote critical thinking. Future work should explore adaptive interfaces and fact-checking integrations to balance informational and social conformity.