Who should set the Standards? Analysing Censored Arabic Content on Facebook during the Palestine-Israel Conflict

Research Background and Issues

• What problems or challenges did the authors identify?
  The authors identified potential systemic bias in Facebook's content moderation process, particularly concerning Arabic-language content related to the Israeli-Palestinian conflict. Some users claimed that their posts were removed even though they did not violate Facebook's Community Standards (FBCS). Additionally, there are significant differences in how people from different cultural backgrounds interpret these standards regarding what content should be removed.

• Why is this issue important?
  Social media serves as a critical channel for political and social expression in modern society. Its growing influence has made the fairness and transparency of content moderation systems a key area of research. As the world's largest social media platform, Facebook's improper content moderation could suppress the voices of marginalized user groups, especially in regions with complex geopolitical contexts, such as the Israeli-Palestinian conflict zone.

• Research Motivation and Related Work
  During the May 2021 Israeli-Palestinian conflict, Arabic users alleged that Facebook's algorithms unfairly removed pro-Palestinian content. While media reports and investigations by human rights organizations have highlighted these issues, there is a lack of systematic empirical analysis to verify such claims. Furthermore, existing research primarily focuses on user perceptions in Western contexts, overlooking significant user groups in the Arab world, which are crucial to understanding the global fairness of content moderation.

Solutions

• What methods or solutions did the authors propose?
  The authors collected 448 deleted Arabic posts and engaged 10 Arabic-speaking annotators to evaluate these posts based on Facebook's Community Standards (FBCS) and their personal opinions. The annotators determined whether the posts violated FBCS and whether they should be removed.

• What is innovative about this solution?
  This study combines quantitative and qualitative methods to explore the fairness of content moderation in cross-cultural contexts from a user-centric perspective. This approach addresses the data gap in prior research, which lacked detailed content analysis, and highlights the impact of cultural differences on the interpretation of community standards.

• What are the implementation steps and key technologies used?

  1. Data Collection: A survey was designed to collect posts voluntarily submitted by Arabic users that had been removed by Facebook.
  2. Data Cleaning and Organization: The collected posts were filtered and cleaned to ensure they included clear text or relevant descriptions.
  3. Annotation Task: High-quality Arabic annotators were recruited via the Appen platform to evaluate each post based on FBCS. Annotators also recorded their personal opinions on whether the posts should be removed.
  4. Data Analysis: The authors analyzed discrepancies between the annotators' judgments and Facebook's decisions, exploring the relationship between these discrepancies and the content's themes.

Research Findings

• What specific findings were obtained?

  1. Low Violation Rate: Only 40.6% of the annotations indicated that the posts violated FBCS, while 71% of the annotators believed these posts should not have been removed.
  2. Significant Thematic Differences: Topics related to Israel and Jewish people were more likely to be deemed violations of FBCS, while pro-Palestinian content was often considered non-violative but was still removed.
  3. Cultural Influence: Annotators' interpretations of the standards were significantly influenced by their cultural backgrounds, particularly in the context of complex geopolitical issues like the Israeli-Palestinian conflict.

• What advantages does this solution have compared to existing ones?
  This study provides data-backed insights through the analysis and comparison of actual content and annotations. This approach is more evidence-based than previous studies, which relied solely on user interviews or social media controversies. It allows for a clearer identification of specific issues in moderation systems, such as algorithmic "false positives" or cultural misunderstandings.

• What were the experimental or evaluation results?
  A large proportion of "wrongly removed" posts were related to pro-Palestinian content, suggesting that Facebook's moderation algorithms or policies may struggle to distinguish between supportive expressions and violations. In contrast, there was higher agreement between annotators and Facebook on clear hate speech, such as insults targeting Jewish people or specific groups.

• Limitations and Future Directions

  1. Limited Cultural Comparison: The study focused solely on Arabic-language content moderation, lacking comparative analysis in other cultural contexts, such as Western societies' interpretations of the same content.
  2. Sample Limitations: The study relied on posts voluntarily submitted by users, resulting in a limited sample size that may not fully capture the scope of the issue.
  3. Future Directions: Future research could include cross-cultural comparative studies, expand the scope to other marginalized groups such as African or Asian users, and extend the research to other platforms like TikTok.

Conclusion

This study reveals potential flaws in Facebook's content moderation system in cross-cultural contexts, particularly when handling content related to the Israeli-Palestinian conflict. The study recommends:

1. Enhancing understanding of complex geopolitical issues.
2. Incorporating local experts or users from marginalized regions to improve moderation quality.
3. Increasing transparency in moderation processes and user appeal mechanisms.
4. Promoting culturally sensitive content moderation policies to achieve global fairness and inclusivity.