Take the Power Back: Screen-Based Personal Moderation Against Hate Speech on Instagram

Paper Title

Take the Power Back: Screen-Based Personal Moderation Against Hate Speech on Instagram

Publication Info

• Topic area: Personal moderation tools for combating hate speech on social media.
• Keywords: Hate speech, personal moderation, Instagram, social media, user-centered design, algorithmic curation, conversational spaces, content filtering, activism, Delphi study.

Background and Problem

• Problem / challenge: Platform-wide moderation systems often fail to address hate speech effectively due to biases, lack of contextual awareness, and a one-size-fits-all approach. Automated systems struggle with detecting nuanced content like counter speech or reappropriated slurs, leading to over- or undermoderation.
• Significance: Hate speech has severe consequences for mental health, public discourse, and identity expression, particularly for activists and marginalized groups. Effective moderation is crucial for creating safer online environments.
• Motivation and related work: Previous research has explored personal moderation tools but has not examined how user needs vary across different parts of social media platforms (referred to as "screens"). This study builds on prior work by investigating screen-specific personal moderation needs and feature preferences.

Solution

• Proposed approach: A three-wave Delphi study to identify screen-specific personal moderation needs and prioritize features for Instagram users targeted by hate speech.
• Novelty:
  1. Introduction of "screens" (e.g., comments, home feed, reels tab) as a conceptual unit for personal moderation.
  2. Empirical investigation of screen-specific personal moderation needs for activists targeted by hate speech.
  3. Prioritization of personal moderation features based on user input.
  4. Design recommendations for screen-sensitive personal moderation tools.
• Procedure and key techniques:
  • Conducted a Delphi study with 40 activists in Germany who experienced hate speech.
  • Participants rated the importance of personal moderation across eight Instagram screens and suggested features.
  • Iterative refinement of feature lists and prioritization through three survey waves.
  • Analysis included quantitative ratings, rankings, and qualitative thematic coding of free-text inputs.

Results

• Concrete findings:
  • Screens rated most important for personal moderation: All Comments (100%), Comments Own Posts (92.5%), Home Feed (87.5%), Reels Tab (87.5%), and All DMs (85.0%).
  • Highly ranked features included Customizing the Response to Detected Hate Speech, Filtering Based on Words, and Differentiating Between Hate and Counter Speech.
  • Conversational screens (e.g., comments, DMs) prioritized input-driven and oversight features, while algorithmically curated screens (e.g., home feed, reels tab) emphasized automation and content-type-specific filtering.
• Advantage over baselines:
  • Provides a screen-specific perspective on personal moderation, addressing gaps in prior research that treated moderation needs uniformly across platforms.
  • Offers actionable design recommendations tailored to user priorities and screen contexts.
• Experiments / evaluation:
  • Three-wave Delphi study with 40 activists, focusing on Instagram.
  • Participants rated and ranked features for eight screens using Likert scales and point allocation tasks.
  • Qualitative analysis of free-text inputs to identify novel feature requests.
• Limitations and future work:
  • Sample limited to activists in Germany, potentially biasing results toward progressive perspectives.
  • Findings specific to Instagram's architecture; generalizability to other platforms requires further study.
  • Future work should explore real-world deployment, cross-platform applicability, and integration with platform and community moderation.

Summary

This study investigates personal moderation needs across different screens on Instagram, focusing on activists targeted by hate speech. Using a Delphi study, it identifies key screens (e.g., comment sections, home feed) and prioritizes features like customizable responses, word filtering, and distinguishing hate speech from counter speech. The findings highlight the importance of tailoring moderation tools to specific screens, balancing user control with usability, and addressing both conversational and algorithmically curated spaces. The work provides actionable design recommendations for screen-sensitive personal moderation tools and sets the stage for future research on integrating personal, platform, and community moderation.