Reimagining Data Work: Participatory Annotation Workshops as Feminist Practice

Paper Title

Reimagining Data Work: Participatory Annotation Workshops as Feminist Practice

Publication Info

• Topic area: Feminist and participatory approaches to data annotation in AI development.
• Keywords: Data annotation, feminist AI, participatory design, ethics of care, pluralism, labor acknowledgment, collective dialogue, gender-related violence, multilingual workshops, media narratives.

Background and Problem

• Problem / challenge: Data annotation in AI development is often exploitative, decontextualized, and undervalues the expertise of annotators, perpetuating power imbalances and colonial structures. Few empirical studies explore how feminist and participatory principles can be enacted in practice.
• Significance: Addressing these issues is crucial for creating AI systems that are ethical, inclusive, and context-sensitive, particularly in domains like media reporting on gender-related violence, which shapes public understanding and policy.
• Motivation and related work: Prior work has proposed frameworks for participatory and feminist AI but lacks concrete examples of implementation. This paper builds on feminist HCI and participatory AI literature, focusing on data annotation as a site for reimagining AI development practices.

Solution

• Proposed approach: Collaborative, multilingual data annotation workshops rooted in feminist epistemology and participatory design principles.
• Novelty:
  1. Demonstrates how workshops can foster dialogue, community, and disrupt knowledge hierarchies in data annotation.
  2. Introduces the concept of "tactical consensus" for balancing pluralism and standardization in annotation.
  3. Proposes "multiple depths of dialogue" to accommodate varying levels of participant engagement.
  4. Explores tensions in acknowledging labor while resisting transactional dynamics.
• Procedure and key techniques:
  1. Organized multilingual workshops with activists, journalists, and researchers to annotate news articles on feminicide.
  2. Iterative design process incorporating feedback to refine taxonomies and annotation practices.
  3. Facilitated breakout rooms for localized and context-specific discussions.
  4. Conducted asynchronous annotation with mechanisms for collective support (e.g., chat groups, weekly meetings).
  5. Compensated participants and acknowledged their contributions in multiple ways.

Results

• Concrete findings:
  • 79 participants from 35 countries contributed to workshops, with 28 continuing in asynchronous annotation.
  • Taxonomies were refined to include culturally specific categories (e.g., stigmatization of victims, overemphasis on perpetrators).
  • Participants annotated 1–60 articles each (average=15.5), with compensation ranging from $40 to $150.
  • Workshops fostered solidarity, critical reflection, and mutual learning among participants.
• Advantage over baselines:
  • Shifted annotation from isolated, decontextualized tasks to collaborative, relational practices.
  • Enabled culturally sensitive and context-specific annotations, addressing limitations of conventional annotation pipelines.
  • Elevated annotators as co-creators, challenging traditional epistemic hierarchies.
• Experiments / evaluation:
  • Workshops included trilingual and language-specific sessions with breakout rooms for localized discussions.
  • Feedback surveys and open coding of transcripts informed iterative improvements.
  • Participants rated the workshops highly for learning and community connection (average 4.6/5 on knowledge gained).
• Limitations and future work:
  • Logistical challenges in compensating participants across global contexts.
  • Difficulty balancing pluralism with the need for standardized taxonomies.
  • Future work will explore methods to document disagreements and incorporate subjectivity into AI models.

Summary

This paper presents a case study of participatory, feminist data annotation workshops focused on news coverage of feminicide. By integrating feminist principles such as care, pluralism, and labor acknowledgment, the workshops transformed annotation into a collaborative and relational process. Key contributions include the concepts of "tactical consensus" for balancing pluralism and standardization, and "multiple depths of dialogue" to accommodate diverse participant engagement. The study highlights the potential of participatory approaches to disrupt extractive AI practices and foster ethical, context-sensitive AI development.