Participation and Division of Labor in User-Driven Algorithm Audits: How Do Everyday Users Work Together to Surface Algorithmic Harms?

Title of the Paper

Participation and Division of Labor in User-Driven Algorithm Audits: How Do Everyday Users Work Together to Surface Algorithmic Harms?

Paper Information

• Subject Area: User-driven algorithm audits and socio-technical interactions
• Keywords: Algorithm audits, user-driven audits, Twitter, image cropping, gender bias, racial bias, user-generated content, user division of labor

Research Background and Problem Statement

• Problem Identified by the Authors: In algorithm audits, the ways in which users identify and discuss algorithmic biases through everyday interactions, as well as the patterns of user participation and division of labor in such audits, remain unclear.
• Significance: Algorithmic bias has significant implications for social justice and technological fairness. Encouraging user participation in audits can uncover more potential harmful algorithmic behaviors.
• Motivation and Related Work:
  • Expert-driven audits are effective but cannot fully identify new biases affecting marginalized groups. User-driven audits, as an emerging approach, help address these limitations.
  • Related work includes how users develop "folk theories" about algorithms and explore algorithmic behaviors through everyday interactions.

Solution

• Method or Solution:
  • The authors conducted a comparative analysis of four user-driven algorithm audit cases: Twitter image cropping, ImageNet Roulette, Portrait AI, and Apple Card.
  • They analyzed user participation patterns and division of labor using Twitter data, employing machine learning to classify user-generated content.
• Innovation: Proposed a classification-based division of labor method that categorizes user-generated content into five roles: hypothesis generation, evidence collection, information dissemination, contextualization, and emotional escalation.
• Implementation Steps:
  1. Data Collection: Using the Twitter API and keyword searches, the authors collected relevant tweets, amassing a total of 17,984 entries.
  2. Data Classification: Developed a machine learning model to classify tweet content and identify key user roles and contributions.
  3. Quantitative and Qualitative Analysis: Analyzed user participation patterns and the distribution of tweet roles across the dataset.

Research Findings

• Specific Results:
  1. Described overall patterns of user participation across the four cases, including user count, peak activity, and duration of engagement.
  2. Defined five user labor roles—hypothesis generation, evidence collection, dissemination, contextualization, and emotional escalation—and analyzed their distribution in different cases.
  3. Highlighted the impact of short-term, high-intensity tweet peaks on the success of user-driven audits.
• Comparison with Existing Solutions: User-driven audits differ from traditional expert audits in their spontaneity, emotional feedback, and potential to raise social awareness, rather than relying on rigorous scientific validation processes.
• Experimental or Evaluation Results: Among the four cases, the Twitter Cropping case had the highest number of tweets and user engagement, followed by ImageNet Roulette and Apple Card.
• Limitations and Future Directions:
  • Limitations: The study was limited to four cases and did not analyze the potential impact of images themselves; it failed to fully capture all characteristics of user-driven audits.
  • Future Directions: Develop tools to support user-driven audits, enhance algorithm transparency, improve user participation and data collection capabilities, and optimize tweet classification models.

Design and Policy Recommendations

• Design Perspective: Improve tools for user discussion and collaboration, address issues of data access and visibility, and design structured support tools to encourage user participation.
• Policy Perspective: Develop policies and regulations to enhance algorithm transparency and accountability, support the formation of small user communities, and facilitate information sharing.

This paper provides new insights and design directions for addressing algorithmic bias through an in-depth study of user-driven algorithm audits, offering significant inspiration for future research and practice.