“Ignorance is not Bliss”: Designing Personalized Moderation to Address Ableist Hate on Social Media

Research Background and Issues

• Identified Challenges: People with disabilities frequently encounter ableist hate and microaggressions on social media. Content moderation mechanisms on these platforms often fail to effectively remove such hateful content and may even mistakenly delete legitimate disability-related posts. Traditional content moderation does not adequately reflect the needs of disabled users, leaving them exposed to harmful content.
• Significance: Ableist content not only harms users' mental health but may also lead to self-censorship and reduced community engagement. This phenomenon is particularly significant for content creators advocating for disability rights. Furthermore, this issue highlights broader social media governance problems, including platforms' inability to adapt to the diverse needs of their users.
• Research Motivation and Related Systems: Existing personalized content moderation tools (e.g., keyword filters, sensitivity sliders, or AI tools) have become increasingly common but lack specialized designs targeting ableist hate. Additionally, there is insufficient research on how users perceive and utilize these tools, especially those addressing identity-related hate.

Solution

• Proposed Approach: This paper employs design probes and participatory methods to propose a personalized content moderation tool aimed at addressing ableist hate. The tool includes two main modules: 1) filter configuration based on categories of ableism; 2) multiple personalized options for presenting hateful content, such as AI rephrasing or content warnings.
• Innovations:
  1. Ableism is subdivided into multiple categories, allowing users to filter content by category.
  2. More flexible options are proposed beyond simply hiding content, such as displaying warnings or using AI to rephrase hateful content.
  3. Special focus is placed on disabled users' trust in the accuracy and transparency of AI tools, introducing an interactive mode for users to train algorithms.
• Implementation Steps and Techniques:
  1. User Research: Conducted interviews and focus group discussions with 23 disabled social media users to explore design requirements for personalized filters.
  2. Design Probes: Explored various interface options (e.g., toggles, sliders, and category selection) for configuring ableism filters and different methods of presenting filtered hateful content (e.g., warning prompts, rephrased text).
  3. Data Analysis: Used thematic analysis to analyze participant feedback and extract key design recommendations.

Research Outcomes

• Specific Findings:
  1. Most users prefer filtering based on ableism categories rather than binary toggles or intensity sliders.
  2. Participants generally favored content warnings over complete blocking or AI rephrasing, as warnings provide greater decision-making autonomy.
  3. Users expressed strong concerns about AI detection accuracy, particularly regarding sensitive language (e.g., reclaimed terms within communities) and contextual understanding.
• Advantages Compared to Existing Solutions:
  • Supports more granular content filtering, better addressing the psychological needs of disabled users.
  • Transparent system design (e.g., "undo incorrect filtering") enhances user trust in the tool.
  • Emphasizes user autonomy, reducing the drawbacks of excessive reliance on algorithms.
• Experimental Results:
  1. User feedback indicated that filter settings based on ableism categories offered the best clarity and usability.
  2. Content warnings achieved a better balance between protecting mental health and managing content.
• Limitations and Future Directions:
  • Limitations: The current study is based on static probe designs and has not tested interactive prototypes; the sample is skewed toward disabled users in North America and Europe, which may not fully represent global perspectives.
  • Future Research:
    1. Develop actual tools and conduct behavioral studies to validate their long-term effectiveness.
    2. Explore personalized filtering needs for other forms of identity-based hate (e.g., racism, sexism).
    3. Investigate how filters can better support content creators' self-expression without limiting community activities or advocacy.

In summary, this study provides valuable insights into building a more inclusive and safe social media environment through in-depth analysis and design exploration. It emphasizes the importance of user decision-making power and transparency in personalized content filtering, laying the groundwork for future technological development and practical implementation.