The Role of Expertise in Effectively Moderating Harmful Social Media Content

Research Background and Issues

• Identified Problems or Challenges:

  1. Social media platforms failed to effectively regulate the dissemination of genocidal content during the Tigray War from 2020 to 2022.
  2. Social media platforms provide limited language support and have not invested in content moderators with deep cultural and linguistic expertise, leading to inefficient content moderation.
  3. Significant disagreements exist among experts regarding the definition and classification of "harmful content" in moderation processes (initial disagreement rate as high as 71%).

• Significance:

  • The Tigray War resulted in an estimated death toll of 600,000 to 800,000 people, considered one of the deadliest genocides of the 21st century. In this context, the spread of harmful social media content exacerbated conflict and violence.
  • Improper content moderation may intensify political violence, genocide, and persecution of minority groups, which is a critical global issue.

• Research Motivation and Related Work:

  • Research Motivation: To explore the specialized skills required for effective moderation of hate and violent content and analyze the sources of expert disagreements and methods for resolving them.
  • Related Work: Existing studies have focused on broader language issues in content moderation (e.g., the Rohingya genocide and the spread of hate speech). However, there is a research gap in content moderation specific to genocidal contexts, especially for non-Western languages.

Solution

• Proposed Methods or Solutions:

  1. Design and conduct a 4-month data annotation study, involving seven experts with linguistic, cultural, and domain knowledge jointly annotating 340 social media posts.
  2. Conduct interviews with 15 commercial content moderators, aiming to understand current moderation practices on social media platforms.
  3. Provide actionable recommendations to improve hate speech annotation and content moderation practices.

• Innovations:

  • The first comprehensive analysis of the complexities of content moderation in a genocidal conflict context, highlighting the interplay of linguistic, cultural, and domain expertise.
  • Exploration of how to enhance moderation consistency and apply more suitable moderation mechanisms for conflict environments by establishing clear disagreement resolution processes.

• Implementation Steps:

  1. Work Planning and Design: Develop a detailed codebook based on moderation rules set by social media platforms.
  2. Data Collection and Cleaning: Extract social media content containing specific keywords (totaling 5.5 million posts) and filter relevant content to ensure quality.
  3. Data Annotation and Disagreement Resolution: Annotate content using the LabelStudio platform and conduct weekly team consultations to resolve disagreements.
  4. Interview Study: Engage in-depth discussions with commercial content moderators to understand current practices and existing issues.
  5. Qualitative and Quantitative Analysis: Measure annotation disagreements using Krippendorf’s Alpha and perform thematic analysis on interview data.

Research Outcomes

• Specific Outcomes:

  1. Clarified the need for deep cultural knowledge, linguistic dialect expertise, and domain specialization in moderating genocidal content.
  2. Found that initial disagreement among experts reached 71%, but team consultations reduced it to 40%, emphasizing the importance of "collaborative discussion to resolve disagreements."
  3. Provided insights into commercial content moderation practices, revealing platforms' prioritization of language skills and superficial cultural awareness while neglecting deeper cultural and domain expertise.
  4. Completed the classification of triggering terms in complex contexts (e.g., genocidal metaphors like "weeds").

• Advantages Compared to Existing Solutions:

  • Focused on resolving disagreements caused by cultural and linguistic differences in moderation, offering a more collaborative and adaptive solution compared to traditional voting-based methods.
  • Not only identified the importance of linguistic and cultural knowledge but also proposed integrating industry domain expertise (e.g., journalists and activists) into the moderation process.

• Experimental or Evaluation Results:

  • The data annotation study showed that after consultation meetings, label consistency significantly improved (Krippendorf's Alpha increased from 0.2 to 0.55), demonstrating the effectiveness of the disagreement resolution process.
  • The interview study revealed moderators' dissatisfaction with current platform processes and policies and their demand for fairer moderation procedures.

• Limitations and Future Directions:

  1. Limitations:
     • The data annotation study involved only seven experts, with a bias toward certain domain and cultural backgrounds; future studies should include more diverse linguistic and cultural participants.
     • Interviews covered only specific regions (primarily African markets), lacking coverage of moderation practices in other global markets.
  2. Future Directions:
     • Expand research to include more linguistic and cultural backgrounds, with a particular focus on low-resource languages.
     • Incorporate perspectives from platform policymakers and market experts to complement those of moderators and annotators.
     • Explore other complex moderation scenarios beyond conflict, such as improving automated hate speech detection tools.