Evaluating Peer Fact-Checking on WhatsApp

Paper Title

Evaluating Peer Fact-Checking on WhatsApp

Publication Info

• Topic area: Misinformation correction in private messaging platforms.
• Keywords: misinformation, WhatsApp, fact-checking, peer corrections, private messaging, social trust, randomized field study, Global South, encrypted platforms, community-based interventions.

Background and Problem

• Problem / challenge: Misinformation on encrypted platforms like WhatsApp spreads rapidly through high-trust networks, but existing top-down interventions (e.g., fact-checking tiplines) fail due to low trust, awareness, and uptake. Organic peer corrections are rare due to social friction.
• Significance: Misinformation on WhatsApp has fueled violence, distorted public health outcomes, and eroded democratic discourse, especially in the Global South, where WhatsApp is a primary information source.
• Motivation and related work: Prior research highlights the ineffectiveness of institutional tiplines and the potential of peer corrections, but lacks real-world evidence on the effectiveness of different correction sources in encrypted environments.

Solution

• Proposed approach: A randomized controlled trial (RCT) comparing the effectiveness of fact-checks delivered by three sources: a national fact-checking organization (T1), a local journalist (T2), and a trusted close contact (T3).
• Novelty:
  1. First ecologically valid randomized test of peer-led fact-checking on WhatsApp.
  2. Empirical insights into how participants interpret corrections in private messaging contexts.
  3. Design implications for leveraging trusted community members as embedded fact-checkers.
• Procedure and key techniques:
  • Conducted a four-arm RCT with 51 participants in southern India.
  • Collected WhatsApp group data and delivered fact-checks attributed to T1, T2, or T3.
  • Measured belief accuracy on corrected misinformation and generalization to novel misinformation.
  • Conducted post-intervention surveys and semi-structured interviews to assess behavioral and attitudinal shifts.

Results

• Concrete findings:
  • Corrections from a trusted close contact (T3) improved belief accuracy by 28.5 percentage points over the control group (p=0.009).
  • Corrections from a local journalist (T2) showed a 24.1 percentage point improvement but were not statistically significant after multiple comparison corrections (p=0.08).
  • Corrections from a national fact-checker (T1) showed no significant effect (p=0.254).
  • No evidence of improved ability to identify novel misinformation across all arms.
• Advantage over baselines:
  • T3 was the only intervention to produce statistically robust improvements in belief accuracy, outperforming T1 and T2.
• Experiments / evaluation:
  • Data collected from 51 participants in Telangana, India, using a privacy-preserving WhatsApp data donation tool.
  • Belief accuracy measured via repeated belief checks during the intervention.
  • Post-surveys assessed generalization, trust changes, and willingness to pay for fact-checking services.
• Limitations and future work:
  • Small sample size (N=51) limits statistical power for between-arm comparisons.
  • Short intervention duration (13 days) precludes assessment of long-term effects.
  • Findings may not generalize beyond the rural, young, male-dominated sample.
  • Future work should explore longitudinal impacts, disentangle the effects of social ties and professional training, and incorporate true information stimuli.

Summary

This study tested the effectiveness of peer-based fact-checking on WhatsApp through a randomized controlled trial in southern India. Corrections from trusted close contacts significantly improved belief accuracy, while corrections from journalists and national fact-checkers did not show robust effects. However, exposure to fact-checks did not enhance participants' ability to identify novel misinformation, suggesting that corrections are context-specific rather than skill-building. The findings highlight the importance of leveraging trusted community members as embedded fact-checkers and suggest that misinformation interventions in encrypted spaces should focus on fostering deliberative mindsets rather than solely transferring verification skills.