"I think this is fair": Uncovering the Complexities of Stakeholder Decision-Making in AI Fairness Assessment

Paper Title

"I think this is fair": Uncovering the Complexities of Stakeholder Decision-Making in AI Fairness Assessment

Publication Info

• Topic area: Stakeholder involvement in AI fairness assessment
• Keywords: AI fairness, stakeholder decision-making, outcome fairness, fairness metrics, fairness thresholds, human-centered AI, participatory design, credit rating, qualitative study, fairness governance

Background and Problem

• Problem / challenge: Current AI fairness assessment practices are predominantly expert-driven, focusing narrowly on technical metrics and legally protected features, often neglecting the perspectives of non-expert stakeholders directly affected by AI decisions.
• Significance: Including diverse stakeholder perspectives is crucial to align AI fairness with societal values, increase transparency, and mitigate public trust issues in AI systems.
• Motivation and related work: Prior research has explored fairness metrics and expert-driven tools but has not adequately incorporated the voices of lay stakeholders. This gap limits the ability of AI systems to reflect broader human-centered fairness concerns.

Solution

• Proposed approach: A participatory fairness assessment framework enabling lay stakeholders to select features, pair features with fairness metrics, and set fairness thresholds in a credit rating scenario.
• Novelty:
  1. Empirical evidence on how lay stakeholders assess AI fairness across features, metrics, and thresholds.
  2. Insights into the importance of unprotected features and the demand for custom fairness metrics.
  3. Design implications for tools that make fairness assessment accessible to non-experts.
• Procedure and key techniques:
  • Conducted a qualitative study with 26 non-expert participants using an interactive prototype system.
  • Tasks included feature selection and ranking, feature-metric pairing with thresholds, and fairness re-ranking.
  • Used the German Credit Dataset and logistic regression to provide a realistic and interpretable AI decision-making context.

Results

• Concrete findings:
  • Participants selected a broad range of features (mean = 6.15, SD = 2.30), including protected (e.g., gender, age) and unprotected features (e.g., telephone, dependents).
  • Counterfactual Fairness (31 selections) and Custom Fairness (28 selections) were the most frequently chosen metrics, while Demographic Parity was not selected by any participant.
  • Fairness thresholds were stricter than typical AI expert standards, often below 6% deviation from optimal fairness.
• Advantage over baselines:
  • Broader feature considerations compared to AI experts, who typically focus on a narrow set of legally protected features.
  • Dynamic and context-specific metric selection and threshold setting, contrasting with the "one-size-fits-all" approach of AI experts.
• Experiments / evaluation:
  • Participants engaged in a credit rating scenario using a prototype system with features like causal graphs, metric explanations, and interactive visualizations.
  • Data collection included think-aloud protocols, screen recordings, and post-task surveys.
• Limitations and future work:
  • Limited to a single stakeholder type and credit rating context; future studies should involve diverse roles and domains.
  • Prototype did not operationalize custom metrics; future tools should support their implementation and integration.
  • Did not explore other fairness perspectives (e.g., procedural fairness, fairness through unawareness).

Summary

This study highlights the complexity of fairness decision-making among lay stakeholders in AI systems. Participants demonstrated broader and more nuanced fairness considerations than AI experts, selecting diverse features, tailoring metrics to features, and setting strict fairness thresholds. The findings underscore the need for human-centered tools that empower stakeholders to articulate and operationalize their fairness expectations. Future work should expand to diverse contexts and stakeholder roles, while addressing the challenges of integrating custom fairness metrics into AI systems.