Towards Effective Human Intervention in Algorithmic Decision-Making: Understanding the Effect of Decision-Makers' Configuration on Decision-Subjects' Fairness Perceptions

Research Background and Problem

• Identified Issues or Challenges: In current algorithmic decision-making systems, human intervention is considered an essential means of safeguarding the rights of those affected by decisions. However, the impact of different decision-maker configurations (e.g., fully automated algorithmic systems versus hybrid systems with human intervention) on the perceived fairness of decision recipients remains unclear. Additionally, existing studies often analyze the characteristics of decision-maker configurations in isolation, neglecting potential interactions between these characteristics.
• Why This Problem Matters: Algorithmic decision-making systems have profound impacts on individual lives and public policy, with fairness being a critical factor for societal acceptance of such systems. If decision recipients lack a sense of fairness regarding these configurations and the processes they generate, it will hinder the responsible application and widespread adoption of algorithmic decision-making systems.
• Research Motivation and Related Work: While some studies have explored the comparison between fully automated decisions and human decisions, finding that human decisions are generally more favorable, research on hybrid decision configurations has yielded conflicting results. Existing studies often overlook the interactions between configuration characteristics. Furthermore, the authors note a close relationship between trust in decision-makers and perceived fairness, which has not been sufficiently explored in algorithmic decision-making systems.

Solution

• Proposed Method or Solution: The authors designed and implemented a mixed-methods study, including qualitative interviews and large-scale quantitative research, to explore how different characteristics of decision-maker configurations (e.g., decision-maker identity, algorithm model type, data source) influence decision recipients' perceptions of decision-makers and their sense of fairness regarding the decision process.
• Innovations:
  • Applied the Ability, Benevolence, and Integrity (ABI) model from organizational psychology to algorithmic decision-making systems, quantifying decision recipients' trust in different decision configurations.
  • Conducted a comprehensive analysis of multiple characteristics (e.g., model type and data source) and potential interactions between these characteristics.
  • Proposed specific design recommendations for more effective human intervention in the public sector.
• Implementation Steps:
  1. Qualitative Interview Study: Conducted interviews with 21 participants to explore which characteristics of decision-maker configurations influence perceptions of ability, benevolence, and integrity in the context of detecting illegal holiday rentals.
  2. Quantitative Experimental Study: Based on interview findings, designed and conducted a large-scale online experiment involving 223 participants to systematically evaluate the impact of configuration characteristics on decision recipients' perceptions and sense of fairness.
  3. Proposed design recommendations: Combined qualitative and quantitative results to provide guidance for designing hybrid decision-maker configurations in the public sector.

Research Findings

• Specific Findings:
  • The authors found that "decision-maker identity" (e.g., fully automated vs. human-algorithm hybrid) significantly influences decision recipients' perceptions of ability, benevolence, and integrity, with hybrid configurations being more favorable than fully automated ones.
  • Perceptions of benevolence were generally low, indicating that even in hybrid configurations, decision recipients still perceive the decision-making process as insufficiently humane.
  • The individual effects of model type and data source were not significant, but their interaction influenced decision recipients' perceptions of decision-maker integrity.
  • Perceptions of ability and integrity were strongly positively correlated with perceptions of fairness.
  • Policy approval levels influenced decision recipients' perceptions of decision-maker integrity, which indirectly affected their sense of fairness through integrity perceptions.
• Advantages Compared to Existing Solutions:
  • Systematically quantified the impact of hybrid decision-maker configurations on perceptions of fairness, providing robust empirical evidence.
  • Proposed a design framework that integrates decision-maker configuration attributes with perceptions of social policies, aiding in the promotion and societal acceptance of algorithms.
• Experimental or Evaluation Results:
  • Hybrid decision configurations significantly improved perceptions of ability and integrity, indirectly enhancing perceptions of fairness in the decision-making process.
  • The effects of model type (probabilistic vs. rule-based) and data source (public vs. non-public data) varied due to interaction, suggesting that designing decision-maker configurations requires more nuanced and comprehensive evaluation.
• Limitations and Future Directions:
  • Limitations: The study focused solely on the illegal holiday rental detection scenario, which may not capture the complexity of all public decision-making contexts. Additionally, participants primarily came from the Global North, potentially limiting cultural applicability.
  • Future Directions: Further validation of the ABI model across diverse decision-making scenarios, particularly in cultural contexts of the Global South; exploration of how to enhance transparency in complex distributed human interventions across every supply chain stage of algorithm design; development of more engineering-oriented tools to guide designers in balancing rule-based models with privacy protection requirements.

The authors' research, grounded in rigorous qualitative and quantitative experiments, provides critical guidance for algorithm design in the public domain while inspiring new reflections on the comprehensiveness and responsibility of human intervention.