Understanding Practices, Challenges, and Opportunities for User-Engaged Algorithm Auditing in Industry Practice

Title of the Paper

Understanding Practices, Challenges, and Opportunities for User-Engaged Algorithm Auditing in Industry Practice

Paper Information

• Subject Area: Human-Computer Interaction, Algorithm Auditing, Social Responsibility of Artificial Intelligence
• Keywords: User-engaged algorithm auditing, Responsible AI, Industry practitioners, Fairness, Bias

Research Background and Problem Statement

• Identified Problems or Challenges:

  • Standard algorithm audits (typically conducted by expert teams) have significant blind spots, failing to uncover certain critical issues that only emerge during real user interactions with algorithmic systems.
  • There is a growing phenomenon of users spontaneously revealing algorithmic biases, but the industry lacks systematic guidance on how to harness the power of such user engagement.
  • The current state of user-engaged algorithm auditing in industry practice and its associated challenges remain largely unexplored.

• Significance:

  • User-engaged algorithm auditing can help uncover issues missed by standard audits, particularly biases related to user identity, cultural background, and specific contextual practices.
  • Exploring how to better integrate user input can aid in building fair and responsible algorithms, thereby advancing societal progress.

• Research Motivation and Related Work:

  • The methodology is motivated by the need to address gaps in current algorithm auditing practices, including understanding users' subjective experiences, overcoming blind spots in audit teams, and leveraging user evidence to drive fairness interventions within companies.
  • This study builds upon prior work on algorithmic bias, user feedback, and crowdsourced auditing, aiming to provide practical guidance for industry applications.

Proposed Solution

• Proposed Methods or Solutions:

  • Investigate the experiences and needs of industry algorithm auditing practitioners through semi-structured interviews and co-design activities.
  • Iteratively design models for user-engaged algorithm auditing, including "developer-led" and "user-led" auditing workflows, as well as templates for user-generated audit reports.

• Innovative Contributions:

  • Uncover unique challenges in user-engaged algorithm auditing that go beyond traditional considerations in human computation and crowdsourcing design.
  • Facilitate future research by designing collaborative activities that provide clearer directions for advancing user-driven algorithm auditing.

• Implementation Steps and Key Techniques:

  1. Step 1: Conduct semi-structured interviews with industry practitioners who have experience in user-engaged algorithm auditing, evaluating motivations and challenges in two phases.
  2. Step 2: Collaboratively design with participants to outline the ideal format of user audit reports and the architecture of auditing workflows, investigating further challenges revealed by the design process.
  3. Data Analysis: Analyze interview transcripts and co-design discussions using qualitative coding and thematic analysis.

Research Findings

• Specific Findings:

  • Identified three primary motivations for user-engaged algorithm auditing: understanding users' subjective experiences, avoiding blind spots in development teams, and obtaining user evidence to drive internal actions.
  • Summarized current small-scale and large-scale user auditing methods adopted by industry practitioners, including user studies, focus groups, crowdsourced feedback, and real-time user feedback.
  • Highlighted key challenges in user-engaged algorithm auditing, such as recruiting appropriate audit users, designing user task guidance, extracting actionable conclusions from bias reports, and enhancing company trustworthiness.

• Advantages Over Existing Methods:

  • User-engaged auditing addresses the shortcomings of expert audits in identifying overlooked issues, particularly in scenarios involving intersectional user identities or complex cultural contexts.
  • By designing workflows that integrate user feedback with developer control, the study provides operational models for future industry practices.

• Experimental or Evaluation Results:

  • Interviews and co-design activities revealed tensions between practical implementation and theoretical goals, such as the risk of relying on "majority votes" in interpreting user audit reports while neglecting feedback from marginalized users.
  • A preference for quantitative data in industry culture conflicts with the detailed and nuanced qualitative findings in user audit reports.

• Limitations and Future Directions:

  • Limitations: Participants were primarily from specific roles or backgrounds (e.g., large U.S. tech companies), which may limit the generalizability of the findings to global industry practices.
  • Future Directions:
    • Develop tools to facilitate the recruitment of diverse and representative user auditors.
    • Explore strategies for long-term and sustained user engagement, including seamless feedback mechanisms and community support.
    • Investigate how to integrate qualitative and quantitative data to derive actionable auditing conclusions.
    • Build platforms with trust mechanisms between users and companies to ensure responsible use of user input by companies and empower users to exert pressure for corporate action.

The findings outlined above provide a comprehensive overview of the current state of user-engaged algorithm auditing and clear directions for future research, offering valuable references for designing effective user interaction mechanisms.