Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability.

Title of the Paper

Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability

Paper Information

• Research Area: Fairness Perceptions in Algorithmic Decision-Making
• Keywords: Algorithmic Decision-Making, Fairness Perceptions, Explainability, Human Oversight, Contestability, Informational Fairness, Procedural Fairness

Research Background and Problem Statement

• Problems and Challenges:

  • Although algorithmic decision-making processes may adhere to certain objective fairness standards, users affected by these decisions may not perceive them as fair.
  • The "inexplicability" and "lack of accountability" of algorithmic systems often conflict with notions of justice, undermining users' trust and acceptance of such systems.
  • Existing research typically examines factors influencing fairness perceptions (e.g., explainability, oversight, and contestability) in isolation, without exploring their interactive relationships.
  • Most prior studies assess "overall fairness" using a unidimensional approach, neglecting more nuanced measurements such as informational fairness and procedural fairness.

• Significance of the Study:

  • Algorithms are widely applied in critical decision-making domains (e.g., loan approvals), where fairness issues are crucial for societal trust and acceptance.
  • Investigating users' perceptions of fairness is essential for improving algorithmic system design to meet users' justice standards.

• Research Motivation and Related Work:

  • Existing studies suggest that explainability, human oversight, and contestability influence users' fairness perceptions, but the specific mechanisms remain unclear.
  • Applying multidimensional fairness evaluation methods from legal psychology to algorithmic decision-making provides theoretical support for further research.

Proposed Solution

• Research Methodology:

  • Conducted a user study (267 participants) simulating a loan approval scenario, manipulating the variables of "explainability," "human oversight," and "contestability."
  • Measured users' perceptions of informational fairness, procedural fairness, and overall fairness.
  • Investigated the impact of task importance (high/low) on fairness perceptions.

• Innovative Contributions:

  • Systematically examined the independent and combined effects of three core factors (explainability, oversight, contestability) on multidimensional fairness perceptions.
  • Proposed new design recommendations to enhance users' perceived fairness.

• Implementation Steps:

  1. Pre-Experiment: Identified user preferences for different explanation styles and contestability mechanisms to inform the main experiment.
  2. Main Experiment: In a loan approval scenario, participants were randomly assigned to groups based on the following variables:
     • Explainability (present/absent)
     • Human oversight (present/absent)
     • Contestability (none, objections providing additional information, objections requesting decision revision)
     • Task importance (high/low)
  3. Data Collection and Analysis: Used factorial ANOVA and multiple linear regression to evaluate the effects of manipulated variables on fairness perceptions.

Research Findings

• Key Findings:

  • Providing explanations significantly enhanced users' perceptions of informational fairness (indicating that users felt decisions were more transparent and comprehensive).
  • Introducing contestability improved users' perceptions of procedural fairness (users were satisfied with the process's ability to be rectified).
  • Users' overall fairness perceptions depended on both informational fairness and procedural fairness, but no significant interaction was found between the two.
  • Human oversight had no significant impact on fairness perceptions and, in some cases, might undermine procedural consistency and debiasing efforts.

• Comparison with Existing Approaches:

  • This study confirmed the positive impact of explainability on informational fairness perceptions and was the first to suggest that human oversight might negatively affect procedural fairness perceptions.
  • By employing multidimensional measurements, this study provided more detailed insights compared to previous unidimensional assessments of "overall fairness," highlighting the distinct effects of various factors on specific fairness dimensions.

• Summary of Experimental Results:

  • Explainability: A key factor in enhancing informational fairness, especially for users with limited AI knowledge.
  • Contestability: Improves procedural fairness, particularly when mechanisms for decision correction are provided.
  • Human Oversight: Does not significantly contribute to users' satisfaction with procedural fairness and may even raise concerns about consistency.

• Limitations and Future Directions:

  • This study was limited to a single scenario (loan approvals), and its relevance to other domains requires further exploration.
  • The use of third-person narratives and single-round interactions may underestimate the direct impact of outcomes on users' perceptions.
  • Broader experiments are needed to account for cross-cultural diversity.
  • Future research should explore how contestability mechanisms can empower users with "procedural voice" and "outcome influence."
  • Encourages the development of more intuitive and user-adaptive "system-level explanations" to enhance transparency and user comprehension.