Request a Note: How the Request Function Shapes X's Community Notes System

Paper Title

Request a Note: How the Request Function Shapes X's Community Notes System

Publication Info

• Topic area: Crowdsourced fact-checking on social media platforms.
• Keywords: Community Notes, misinformation, fact-checking, social media, crowdsourcing, user requests, political bias, content moderation, GPT, polarization.

Background and Problem

• Problem / challenge: Despite the promise of X's Community Notes system in addressing misinformation, its scalability and coverage remain limited. The impact of the recently introduced "Request Community Note" feature on what gets fact-checked, who participates, and the quality of notes is unclear.
• Significance: Understanding the effectiveness of the request feature is critical for improving the scalability and responsiveness of crowdsourced fact-checking systems, especially in combating misinformation on social media.
• Motivation and related work: Prior research has shown that Community Notes can reliably identify misleading posts and reduce their spread. However, limitations include slow response times, low display rates of notes, and restricted contributor participation. The request feature aims to address these issues by enabling broader user participation, but its effects have not been systematically studied.

Solution

• Proposed approach: A comprehensive empirical analysis of the "Request Community Note" feature on X, using a dataset of 98,685 requested posts and their associated notes.
• Novelty:
  1. First empirical evaluation of the request feature's impact on content selection, contributor behavior, and note quality.
  2. Identification of distinct selection patterns between requestors and contributors.
  3. Analysis of the helpfulness and polarization of request-fostered notes compared to other notes.
• Procedure and key techniques:
  • Logistic regression models to analyze the likelihood of posts receiving notes based on content and author characteristics.
  • Timing analysis to assess the role of requests in fostering note creation.
  • Replication of X’s note selection algorithm to evaluate note helpfulness and polarization.
  • Use of GPT-4.1 to classify post content and estimate misleadingness.

Results

• Concrete findings:
  • 53.6% of requested posts received notes, but only 12.1% were likely influenced by the request feature.
  • Posts with higher misleadingness, entertainment or finance topics, and media content were more likely to receive notes, while political and scientific posts were less likely.
  • Request-fostered notes were rated as more helpful (mean helpfulness = 0.231) and less polarized (mean polarization = 0.424) than other notes.
• Advantage over baselines:
  • Request-fostered notes outperformed both request-related and writer-only notes in helpfulness and polarization metrics.
  • Requests shifted top writers’ focus toward politically salient and misleading posts, improving note quality for high-stakes content.
• Experiments / evaluation:
  • Dataset: 98,685 requested posts, 5,888,351 requests, and 1,787,609 notes.
  • Metrics: Likelihood of receiving notes, note helpfulness, and polarization.
  • Validation: Manual and expert-verified validation of GPT annotations.
• Limitations and future work:
  • Observational data limits causal inference.
  • Potential biases in GPT-based annotations.
  • Difficulty in perfectly distinguishing request-fostered notes from other notes.
  • Future work should explore causal relationships, improve annotation accuracy, and analyze individual-level contributor behavior.

Summary

This study evaluates the impact of X's "Request Community Note" feature on its Community Notes system. It finds that while requests have a limited direct effect on note creation (12.1% of requested posts received request-fostered notes), they effectively guide contributors toward politically salient and misleading content. Request-fostered notes are rated as more helpful and less polarized than other notes, reflecting contributors’ selective focus on high-risk posts. These findings highlight the potential of the request feature to enhance the scalability and quality of crowdsourced fact-checking, though challenges remain in addressing delays, polarization, and reliance on a small group of top contributors.