Beyond Community Notes: A Framework for Understanding and Building Crowdsourced Context Systems for Social Media

Paper Title

Beyond Community Notes: A Framework for Understanding and Building Crowdsourced Context Systems for Social Media

Publication Info

• Topic area: Crowdsourced systems for content moderation on social media.
• Keywords: Crowdsourced Context Systems, misinformation, content moderation, social media, Community Notes, HCI, design space, transparency, user informedness, algorithmic curation.

Background and Problem

• Problem / challenge: Social media platforms face challenges in addressing misinformation effectively. Traditional fact-checking methods are resource-intensive, slow, and often criticized for bias. Crowdsourced Context Systems (CCS) offer a potential alternative but lack a comprehensive understanding of their design, functionality, and impact.
• Significance: CCS have the potential to reshape the information ecosystem by providing scalable, user-driven content moderation tools that could reduce misinformation spread and increase user informedness.
• Motivation and related work: Previous research has explored content moderation, professional fact-checking, and crowdsourced fact-checking systems. However, CCS remain underexplored as a distinct system class, particularly in terms of their design, implementation, and normative implications.

Solution

• Proposed approach: The paper proposes a framework for understanding and designing CCS, consisting of a theoretical model and a design space with six key aspects: participation, inputs, curation, presentation, platform treatment, and transparency.
• Novelty:
  1. A theoretical model defining CCS and distinguishing them from prior systems.
  2. A design space taxonomy outlining key implementation decisions across six aspects.
  3. A systematic literature review (n=56) of CCS research, focusing on Twitter/X’s Community Notes.
  4. An analysis of real-world CCS implementations across major platforms (Twitter/X, Meta, YouTube, TikTok).
• Procedure and key techniques:
  • Conducted a systematic literature review using PRISMA methodology.
  • Performed inductive thematic analysis and affinity diagramming on CCS design features from public documentation.
  • Synthesized findings into a theoretical model and a detailed design space.

Results

• Concrete findings:
  • CCS can reduce misinformation spread and increase user informedness but often produce notes too slowly to address viral content effectively.
  • Bridging algorithms mitigate bias but fail to address polarizing content effectively, with only ~10% of suggested notes published on X and ~6% on Meta.
  • Users perceive CCS notes as credible, sometimes more so than professional fact-checks.
• Advantage over baselines: CCS are faster and cheaper than professional fact-checking and leverage user contributions to scale content moderation. However, they face trade-offs in speed, coverage, and quality.
• Experiments / evaluation:
  • Literature review categorized studies into four areas: system impact (n=24), contributor behavior (n=16), system design (n=10), and user perceptions (n=6).
  • Analysis of CCS implementations revealed significant design variations across platforms, particularly in participation, curation, and transparency.
• Limitations and future work:
  • Limited focus on Twitter-style CCS may have excluded relevant earlier work.
  • Analysis of platform implementations relied on public documentation, which may not fully reflect deployed systems.
  • Future work should explore underexamined design aspects (e.g., participation incentives, AI integration) and improve transparency for holistic evaluation.

Summary

This paper introduces a framework for understanding and designing Crowdsourced Context Systems (CCS), which are emerging tools for addressing misinformation on social media. By conducting a systematic literature review and analyzing real-world implementations, the authors identify a theoretical model and a design space with six key aspects. CCS show promise in reducing misinformation and increasing user informedness but face challenges in speed, coverage, and fairness. The framework provides a foundation for future HCI research to optimize CCS design and integration into broader content moderation ecosystems.