Risk, Data, Alignment: Making Credit Scoring Work in Kenya

Paper Title

Risk, Data, Alignment: Making Credit Scoring Work in Kenya

Publication Info

• Topic area: Algorithmic credit scoring and its sociotechnical implications in Kenya.
• Keywords: Credit scoring, Kenya, alternative data, machine learning, financial inclusion, risk, uncertainty, alignment, fintech, privacy.

Background and Problem

• Problem / challenge: Existing credit scoring systems in Kenya face challenges such as limited data coverage, high default rates, and reliance on proprietary data infrastructures, which exclude large segments of the population from access to credit.
• Significance: Credit scoring impacts financial inclusion, access to resources, and broader economic participation, especially in regions with limited traditional credit data.
• Motivation and related work: Prior studies in HCI and ML have explored automated decision systems and their social impacts, but most focus on Western contexts. In Kenya, unique credit cultures, monopolistic data infrastructures, and regulatory gaps create distinct challenges for algorithmic credit scoring.

Solution

• Proposed approach: The paper investigates how algorithmic credit scoring systems are developed and operationalized in Kenya through sociotechnical and institutional practices, focusing on the role of alternative data and alignment.
• Novelty:
  1. Analysis of alternative data collection methods, including technical workarounds like SMS scraping.
  2. Examination of risk construction and its entanglement with uncertainty in epistemic, modeling, and contextual forms.
  3. Introduction of alignment as a two-way translation: making models safe for worlds and reshaping worlds to fit model assumptions.
  4. Ethnographic insights into the practices of startups, banks, telcos, and regulators in Kenya.
• Procedure and key techniques:
  • Ethnographic fieldwork conducted over nine months in Nairobi, including participant observation, interviews, and analysis of datasets.
  • Documentation of practices such as feature engineering, risk formulation, and model evaluation.
  • Exploration of the sociotechnical dynamics of credit scoring, including cultural and political influences.

Results

• Concrete findings:
  • Alternative data sources, such as SMS logs, enable credit scoring for underserved populations but introduce privacy and ethical concerns.
  • Risk is constructed through expert guessing, feature engineering, and behavioral assumptions, often leading to biases and surveillance.
  • Default rates in digital lending remain high (e.g., Hustler Fund’s 68.3% non-performing loan rate), challenging the efficacy of credit scoring systems.
• Advantage over baselines:
  • Hybrid models incorporating alternative data outperform traditional scores by up to 80% in predictive accuracy, according to developers.
  • Expanded data sources increase coverage for previously excluded groups, such as MSMEs and rural populations.
• Experiments / evaluation:
  • Evaluation of credit scoring models using metrics like the Gini coefficient, with acceptable ranges varying by model type (e.g., 0.3 for application models, 0.6 for behavioral models).
  • Observations of model drift and the need for regular updates to maintain predictive power.
• Limitations and future work:
  • Legal and ethical ambiguities surrounding alternative data collection and usage.
  • Persistent gaps between claimed model performance and real-world outcomes.
  • Need for further research on balancing privacy, inclusion, and developmental goals.

Summary

This paper investigates the sociotechnical and institutional practices of algorithmic credit scoring in Kenya, focusing on the role of alternative data and alignment. It highlights how risk is constructed and entangled with uncertainty, complicating predictive accuracy and raising ethical concerns. Through ethnographic fieldwork, the authors document how startups, banks, and regulators navigate technical, cultural, and political challenges to make credit scoring work. The findings underscore the provisional nature of alignment, where models and worlds are mutually reshaped, with uneven consequences for privacy, surveillance, and financial inclusion. The study calls for deeper inquiry into the normative and practical tensions inherent in high-stakes decision systems like credit scoring.