Understanding Contestability on the Margins: Implications for the Design of Algorithmic Decision-making in Public Services

Title of the Paper

Understanding Contestability at the Margins: Implications for Algorithmic Decision-Making in Public Services

Citation Information

• Authors: Naveena Karusala, Sohini Upadhyay, Rajesh Veeraraghavan, Krzysztof Gajos
• Conference: CHI Conference on Human Factors in Computing Systems (CHI ’24)
• Publication Date: May 11–16, 2024
• Keywords:
  • Algorithmic Decision-Making
  • Public Services
  • Contestability
  • India
  • United States
  • Accompaniment
  • Explainability
  • Marginalized Populations
  • Information Opacity
  • Procedural Fairness

Research Background and Problem Statement

• Problem Identified: This study finds that, globally, despite increasing attention from policymakers and researchers on contestability in algorithmic decision-making, these discussions have yet to adequately reflect the lived experiences of marginalized groups. Particularly in non-Western countries and disadvantaged communities, there is a lack of detailed understanding of how to implement effective contestability mechanisms.
• Significance: Algorithmic decision-making is widely applied in various public domains (e.g., housing, land resources, and welfare distribution), which may exacerbate existing inequalities. Without effectively designed mechanisms for contestation and redress, the fundamental rights of marginalized populations may be further eroded.
• Motivation and Related Work: This study takes real-world contestation scenarios in socio-technical systems as a research opportunity. By analyzing cases in public housing services in India and the United States, it seeks to uncover the broader social contexts and the impact of contestability mechanisms on design and policy.

Proposed Solution

• Proposed Approach:
  • Drawing on the concept of "accompaniment" from the healthcare field, the study proposes integrating a continuous, open, and accountable accompaniment model into the design of algorithmic decision-making systems.
  • In practical applications, it explores how multi-stakeholder collaboration (e.g., NGOs, legal advocates) can assist marginalized populations in making better decisions, contesting outcomes, and achieving fair results in complex decision-making processes.
• Innovative Contributions:
  • This is the first study to propose the feasibility of extending the "accompaniment" concept to public services, using it to help marginalized groups identify and respond to injustices caused by algorithmic decisions.
  • By examining specific cases (the U.S. housing crisis and land use disputes in India), the study refines contestability models across different political and social contexts.
• Implementation Steps:
  • Conduct qualitative research using in-depth interviews and observational methods to study contestation processes in public services in rural India (e.g., the Irula community) and urban U.S. (e.g., the Boston area).
  • Organize data analysis using coding and cross-comparison methods to uncover the roles and participation of different actors (applicants, community coordinators, government agencies) in public service contestation.
  • Summarize findings to provide specific recommendations for improving the design of contestable decision-making systems.

Research Outcomes

• Key Findings:
  • Revealed how the lack of information transparency and the complexity of implementation mechanisms in digital processes exacerbate challenges for marginalized groups in accessing public resources.
  • Highlighted the critical role of accompaniers (e.g., NGOs, lawyers, community leaders) in interpreting, addressing, and guiding actions in contestation processes.
  • Proposed specific recommendations for optimizing the design of public service algorithms (e.g., enhancing communication transparency, ensuring human oversight) to reduce procedural and outcome inequities.
• Comparison with Existing Solutions:
  • This study moves beyond the traditional focus on transparency and adversarial explainability in algorithmic decision-making discussions, shifting attention to the social support, knowledge transfer, and influence strategies required in the contestation process.
  • In contrast to studies rooted in exclusively Western contexts, this research incorporates practices from the Global South, offering insights for governance in developing countries.
• Experimental and Evaluation Results:
  • In India, the study found that community coordinators and legal advocates are central to ensuring the effectiveness of contestation, though they are constrained by limited resources and cross-departmental coordination challenges.
  • In the U.S., despite having a comprehensive public housing application mechanism, factors such as language barriers, legal knowledge, and cultural differences lead immigrant applicants to passively forgo contestation.
• Limitations and Future Directions:
  • Limitations:
    • The study focuses on scenarios where algorithms were not directly utilized, providing contextual references but not validating the fundamental changes that algorithmic intervention might bring to behavioral dynamics.
    • Data is concentrated on a limited number of case studies, which is insufficient for comprehensive policy recommendations.
  • Future Directions:
    • Delve deeper into the development and deployment of specific algorithmic tools to study how they can collaborate with social support systems.
    • Explore the potential structural role of algorithmic governance in non-Western countries (e.g., promoting grassroots accountability through data transparency).