Sociotechnical Challenge Modeling: A Design Method for Responsible AI in Healthcare and Social Welfare

Paper Title

Sociotechnical Challenge Modeling: A Design Method for Responsible AI in Healthcare and Social Welfare

Publication Info

• Topic area: Responsible AI design methods for healthcare and social welfare.
• Keywords: Responsible AI, sociotechnical challenges, machine learning, healthcare, social welfare, design methods, participatory design, interdisciplinary collaboration, mitigation planning, HCI.

Background and Problem

• Problem / challenge: Machine learning (ML) tools in healthcare and social welfare face sociotechnical challenges such as rigid workflows, incomplete data, structural discrimination, and mismatched assumptions with professional values. Practitioners lack methodological support to identify and mitigate these challenges effectively.
• Significance: Addressing sociotechnical challenges is critical for the ethical, effective, and efficient deployment of ML tools in resource-constrained sectors like healthcare and social welfare.
• Motivation and related work: Existing Responsible AI (RAI) methods often focus on technical design or normative considerations but lack systematic approaches for addressing sociotechnical challenges. Previous methods like AI Failure Cards and AI Mismatch Approach have explored failure anticipation but do not integrate the sociotechnical perspective comprehensively.

Solution

• Proposed approach: Sociotechnical Challenge Modeling (STCM), a workshop-based design method to help practitioners identify and address sociotechnical challenges in ML deployments.
• Novelty:
  1. A structured design method integrating a sociotechnical framework with practical tools like challenge cards and countermeasure worksheets.
  2. Empirical evaluation of STCM in real-world healthcare and social welfare settings.
  3. Focus on interdisciplinary collaboration and resource-constrained environments.
  4. Materials freely available for adaptation and use.
• Procedure and key techniques:
  1. Challenge Prioritization: Participants use Sociotechnical Challenge Cards to identify and rank challenges based on likelihood and severity.
  2. Countermeasure Planning: Participants brainstorm and select mitigation strategies using Countermeasure Worksheets, combining predefined and novel ideas.
  3. Materials include physical cards, worksheets, facilitator guidance, and workshop templates for ease of adoption.

Results

• Concrete findings:
  • STCM revealed novel sociotechnical challenges and shifted participants' perspectives from technical to sociotechnical interdependencies.
  • Examples on the challenge cards were highly valued for fostering understanding and discussion.
  • Predefined countermeasures were seen as overly prescriptive, leading to revisions for more open-ended brainstorming.
  • Participants reported incorporating STCM outputs into project planning.
• Advantage over baselines:
  • STCM provided a systematic framework compared to ad-hoc or blank-slate workshops.
  • Enabled interdisciplinary collaboration and shared understanding among diverse stakeholders.
• Experiments / evaluation:
  • Field experiment with 26 participants across two UK organizations (ASC01 and HC01).
  • Activities included Challenge Prioritization and Countermeasure Planning, followed by interviews and observations.
  • Data analyzed using the Framework Method with 113 open codes across 6 themes.
• Limitations and future work:
  • Limited evaluation of Countermeasure Planning at HC01 due to time constraints.
  • Focused on discriminative ML tools; applicability to generative AI remains untested.
  • Domain-specific to healthcare and social welfare; broader generalizability requires adaptation.
  • Future work includes longitudinal studies and deployment in additional sectors.

Summary

Sociotechnical Challenge Modeling (STCM) is a novel design method to help practitioners in healthcare and social welfare address the sociotechnical challenges of ML deployments. Through structured workshops, STCM enables interdisciplinary teams to identify, prioritize, and mitigate challenges using tangible tools like challenge cards and worksheets. Field experiments demonstrated its utility in fostering collaboration, revealing novel challenges, and integrating outputs into project planning. While domain-specific and focused on discriminative ML tools, STCM offers a transferable framework for Responsible AI design in resource-constrained settings. Future work will explore its applicability to generative AI and other sectors.