Empowering Stakeholders with Participatory Auditing of Predictive AI: Perspectives from End-Users and Decision Subjects without AI Expertise

Paper Title

Empowering Stakeholders with Participatory Auditing of Predictive AI: Perspectives from End-Users and Decision Subjects without AI Expertise

Publication Info

• Topic area: Participatory AI auditing for non-experts in predictive AI applications.
• Keywords: Participatory auditing, predictive AI, end-users, decision subjects, AI accountability, co-design workshops, health applications, AI transparency, impact discovery, AI metrics.

Background and Problem

• Problem / challenge: Current AI auditing practices are dominated by technical experts and lack tools and processes to involve non-expert stakeholders such as end-users and decision subjects. This limits the identification of socio-technical impacts and potential blind spots.
• Significance: Broadening the scope of AI auditing to include non-experts can enhance accountability, uncover overlooked impacts, and contribute to responsible AI development.
• Motivation and related work: While existing AI auditing tools focus on technical aspects like fairness and robustness, they are inaccessible to non-experts. Recent research highlights the value of participatory auditing but lacks structured tools and methodologies to support non-expert involvement.

Solution

• Proposed approach: Participatory AI auditing process and tool design to empower non-experts (end-users and decision subjects) to contribute meaningfully to AI audits.
• Novelty:
  1. Demonstrating the value of participatory auditing from the perspectives of non-experts.
  2. Defining a structured participatory auditing process across all stages of AI development.
  3. Identifying information and support needs of participatory auditors.
  4. Proposing design directions for tools that support participatory auditing.
• Procedure and key techniques:
  • Conducted nine co-design workshops with 17 participants (patients, teachers, and parents) using two health-related AI applications: SPARRA (hospital readmission risk prediction) and SAM (child attachment style prediction).
  • Iterative workshop design based on a four-stage auditing process: Impact Discovery, Standards Identification, Performance Analysis, and Audit Communication.
  • Developed and refined prototypes for participatory auditing tools based on participant feedback.

Results

• Concrete findings:
  • Participants identified both positive and negative impacts of AI applications, including impacts not covered by existing taxonomies.
  • Participants struggled with creating metrics for impacts but emphasized the need for tools to guide this process.
  • Participants requested accessible information about AI applications, including goals, data sources, performance, and ethical considerations.
• Advantage over baselines:
  • Broadened scope of auditing to include socio-technical impacts and early-stage involvement.
  • Highlighted the need for tools that support non-experts throughout the entire auditing process, unlike existing tools focused on technical metrics.
• Experiments / evaluation:
  • Workshops involved diverse stakeholder groups (patients, teachers, parents) and focused on two AI applications with different levels of transparency (opaque-box and translucent-box audits).
  • Activities included impact assessment, metric creation, and tool design feedback.
• Limitations and future work:
  • Limited participant numbers and diversity, especially for SAM workshops.
  • Iterative workshop design prevents direct comparison of results across groups.
  • Future work includes developing and evaluating comprehensive participatory auditing tools and extending the approach to generative AI applications.

Summary

This paper addresses the challenge of involving non-experts in AI auditing by proposing a participatory auditing process and tool design. Through nine co-design workshops with patients, teachers, and parents, the study demonstrated the value of participatory auditing in identifying overlooked impacts and emphasized the need for accessible tools to support non-experts. Key findings include the importance of early-stage auditing, the inclusion of both positive and negative impacts, and the need for guidance in metric creation. The proposed approach lays the groundwork for empowering stakeholders to contribute to responsible AI development, with future work focusing on tool implementation and application to generative AI systems.