Simple changes to content curation algorithms affect the beliefs people form in a collaborative filtering experiment

Paper Title

Simple changes to content curation algorithms affect the beliefs people form in a collaborative filtering experiment

Publication Info

• Topic area: Effects of algorithmic content curation on user beliefs and behaviors
• Keywords: content curation, social media algorithms, belief accuracy, consensus, engagement-based ranking, bridging-based ranking, intelligence-based ranking, polarization, misinformation, collaborative filtering

Background and Problem

• Problem / challenge: Social media algorithms often prioritize engagement metrics, which can lead to polarization, misinformation, and misalignment with user welfare. There is limited controlled experimental evidence on how alternative algorithms might affect user beliefs and behaviors.
• Significance: Understanding how algorithmic changes influence belief formation is crucial for designing social media systems that promote prosocial outcomes like consensus and belief accuracy, rather than polarization and misinformation.
• Motivation and related work: Prior studies have highlighted the negative impacts of engagement-based algorithms, but evidence on alternative approaches like bridging-based and intelligence-based ranking is sparse. This paper builds on proposals for prosocial media by experimentally testing these alternative algorithms.

Solution

• Proposed approach: The study evaluates three content curation algorithms — engagement-based, bridging-based, and intelligence-based ranking — in a controlled collaborative filtering experiment to assess their effects on belief accuracy and consensus.
• Novelty:
  1. Empirical evidence on how algorithmic content curation affects belief formation and consensus.
  2. Development of alternative ranking algorithms using only basic engagement signals and user demographics.
  3. A low-cost, platform-independent experimental method to test algorithmic effects.
• Procedure and key techniques:
  • Study 1: Created a content inventory of argumentative posts by collecting engagement data (upvotes, downvotes) from 500 participants with balanced political leanings.
  • Study 2: Exposed 1,000 participants to algorithmically ranked feeds (random, engagement-based, personalized engagement-based, bridging-based, intelligence-based) and measured belief updates, consensus, and accuracy.
  • Dependent variables included belief variance (consensus), collective error, and individual error.

Results

• Concrete findings:
  • Personalized engagement-based ranking led to less consensus and less accurate beliefs compared to other algorithms, despite being perceived as more insightful.
  • Bridging-based ranking partially promoted consensus, especially in reducing the belief gap between liberals and conservatives.
  • Intelligence-based ranking partially improved collective and individual belief accuracy but did not outperform non-personalized engagement-based ranking in all cases.
• Advantage over baselines:
  • Bridging-based ranking reduced the liberal-conservative belief gap more effectively than random and personalized engagement-based ranking.
  • Intelligence-based ranking improved belief accuracy more than random and personalized engagement-based ranking.
• Experiments / evaluation:
  • Study 1: Developed a content inventory of 72 posts across six topics (three subjective, three objective).
  • Study 2: Tested five ranking algorithms on belief updating tasks with 1,000 participants, analyzing changes in consensus and accuracy within 10,000 statisticized groups.
• Limitations and future work:
  • Small, artificial content inventory; results may not generalize to real-world social media.
  • Single exposure design; effects on long-term belief durability were not tested.
  • Lack of social cues (e.g., user identities, prior engagement metrics) in the experimental setup.
  • Future work should use larger, more realistic content inventories and test longitudinal effects.

Summary

This study demonstrates that simple changes to content curation algorithms can significantly influence belief accuracy and consensus in a controlled experimental setting. While personalized engagement-based ranking — common on social media platforms — led to polarized and less accurate beliefs, bridging-based and intelligence-based ranking showed potential for promoting prosocial outcomes. The findings highlight the feasibility of designing algorithms that align with user welfare using basic engagement signals and demographic data. The proposed experimental method offers a low-cost, platform-independent approach to studying algorithmic effects, complementing existing methodologies. Future research should explore these algorithms in more realistic and longitudinal contexts.