(Beyond) Reasonable Doubt: Challenges that Public Defenders Face in Scrutinizing AI in Court

Title of the Paper

(Beyond) Reasonable Doubt: Challenges that Public Defenders Face in Scrutinizing AI in Court

Paper Information

• Subject Area: Human-Computer Interaction, Algorithmic Fairness, Application of Artificial Intelligence in Judicial Systems
• Keywords: Criminal Justice, Algorithmic Decision Systems, Artificial Intelligence, Contestability, Performance Evaluation

Research Background and Issues

• Identified Problems or Challenges:

  • Public defenders in the U.S. criminal legal system face challenges in assessing and questioning the reliability of government-used computer forensic software (CFS) for convictions and incarcerations.
  • Judges and juries often exhibit uncritical attitudes toward algorithm-driven decision systems.
  • There is a lack of sufficient resources and expertise to conduct detailed reviews of CFS usage.

• Significance:

  • Algorithm-driven judgments occupy high-risk decision-making positions, where errors can lead to severe judicial consequences, such as wrongful convictions.
  • A scientific approach to investigation and questioning is crucial to ensuring judicial fairness and reducing technological misuse.

• Research Motivation and Related Work:

  • Motivation: To clarify the limitations in practices for contesting the use of CFS and explore ways to make these systems more accountable and transparent.
  • Related Work: Previous studies have focused on theoretical frameworks and fairness in auditing algorithms but lack research grounded in actual judicial contexts.

Solutions

• Proposed Methods or Solutions:

  • Conduct interviews with 17 public defenders and relevant technical experts to analyze the challenges faced by defenders in evaluating and contesting CFS.
  • Provide recommendations to support public defenders, including improving performance testing designs, offering collaborative tools, and enhancing related policies.

• Innovative Aspects:

  • Investigates issues of transparency and contestability of algorithmic systems from the perspective of public defenders, focusing on the practical performance of algorithms in judicial applications.
  • Combines technical performance evaluation with social and institutional contexts, proposing interdisciplinary collaboration and skill-sharing improvements.

• Implementation Steps and Key Techniques:

  1. Interview Collection: Conduct semi-structured interviews with members of the public defender community.
  2. Data Analysis: Use inductive qualitative analysis methods to extract themes.
  3. Framework Development: Based on findings, propose policy recommendations and explore institutional support for public technological capacity.

Research Outcomes

• Specific Findings:

  • Identified three major challenges faced by public defenders in contesting CFS: difficult developer and user strategy operations, uncritical courtroom attitudes, and reliance on expert knowledge.
  • Found significant limitations in performance evaluation designs, such as mismatches with real judicial scenarios.
  • Highlighted the importance of interdisciplinary collaboration and policy interventions, providing guidance for future performance evaluation design and communication.

• Comparative Advantages Over Existing Solutions:

  • Offers in-depth insights based on real judicial scenarios rather than purely theoretical discussions.
  • Specifically addresses the issue of how public defenders evaluate algorithmic reliability, tailoring solutions to high-risk contexts.

• Experimental or Evaluation Results:

  • Public defenders face multifaceted barriers from technical, social, and institutional dimensions, limiting their ability to question CFS reliability.
  • Experiments demonstrate that introducing independent performance evaluations and participatory evaluation designs can significantly support the process of contesting algorithms.

• Limitations and Future Directions:

  • Limitations: The study focuses on the perspective of public defenders and could be expanded to include other judicial stakeholders (e.g., judges, prosecutors, defendants).
  • Future Directions:
    1. Investigate how to more broadly integrate public intuition and technical expertise in designing performance evaluations.
    2. Explore the intersection between judicial perceptions of algorithm reliability and data analysis processes.

Conclusion

Through empirical research, the paper reveals the complex challenges public defenders face when contesting government algorithmic tools (e.g., facial recognition, genetic profiling tools) and proposes recommendations for designing fairer and more contestable algorithmic system performance tests. It provides significant theoretical and practical support for socio-technical interdisciplinary research, offering critical reference points for ensuring the accountable application of algorithms in judicial systems.