The Situate AI Guidebook: Co-Designing a Toolkit to Support Multi-Stakeholder, Early-stage Deliberations Around Public Sector AI Proposals

Document Title

The Situate AI Guidebook: Co-Designing a Toolkit to Support Multi-Stakeholder Early-stage Deliberations Around Public Sector AI Proposals

Document Information

• Topic Area: Early-stage design and decision support for public sector AI tools
• Keywords: Public sector AI, participatory design methods, responsible AI, technology governance and policy, AI ethics toolkit

Research Background and Issues

Key Challenges and Problem Identification

• Public sector institutions are rapidly deploying AI systems to support critical decisions, such as child welfare, criminal justice, and public health. However, many AI systems fail in practice, leading to wasted resources and negative societal impacts.
• These failures often trace back to the early stages of the AI lifecycle (e.g., problem formulation and design ideation), where many AI tools face inherent risks of failure from the outset.
• Existing AI ethics toolkits are primarily designed for industrial contexts and often assume that the decision to develop a particular AI tool has already been made. This results in a lack of systematic decision support for early-stage deliberations on whether and how to develop AI tools.

Importance and Research Motivation

• The impact of public sector AI systems involves historically marginalized communities affected by systemic injustices, making responsible design particularly critical.
• Designing a toolkit to support early-stage deliberations among multi-stakeholder groups (e.g., policymakers, AI developers, community representatives) regarding AI development decisions can help avoid unnecessary social and technical risks.

Related Work

• Existing AI ethics toolkits (e.g., Model Cards and algorithmic impact assessment tools) primarily serve as communication tools but are often insufficient to support collaborative reflection in early stages.
• Literature highlights that technical solution design often implicitly embeds policy decisions, leading to conflicts with community values. Current toolkits rarely focus on enabling multi-stakeholder collaboration, especially in complex, high-risk social contexts.

Solution

Overview of Methods or Solutions

• Situate AI Guidebook is a co-designed toolkit aimed at supporting public sector institutions in making thoughtful early-stage decisions about whether to develop or deploy specific AI tools.
• Using an iterative co-design approach, the toolkit was developed in collaboration with institutional leaders, AI developers, frontline staff, and community advocates. It includes guided questions, external resource references, decision communication templates, and process design recommendations.

Innovations

• Multi-topic Guide: Covers discussions on goals and purposes, social and legal considerations, data and modeling constraints, and organizational governance factors, with a total of 132 questions.
• Modular Content Design: Questions are tiered to support flexible configuration and application, allowing the toolkit to be tailored to institutional needs.
• Design Principles:
  • Encourage reflective discussions (e.g., discouraging simple "yes or no" answers).
  • Ensure practical operability of toolkit processes (e.g., easy integration with existing institutional workflows).

Implementation Steps and Technical Details

1. Semi-structured Interviews: Discuss challenges and needs in AI design and decision-making with research participants.
2. Iterative Co-design: Participants propose critical questions and validate existing ones on collaborative platforms (e.g., Mural).
3. Toolkit Development and Refinement: Integrate proposed questions and optimize content based on interviews and design sessions.
4. Suggested Process Design: Includes a four-stage decision process: identifying topics and participants, selecting and discussing questions, summarizing and defining action items, and public reporting and improvement.

Research Outcomes

Specific Outcomes

• Toolkit Outcomes:

  • The Situate AI Guidebook includes 132 questions categorized into four main topics: "Goals and Purposes," "Social and Legal Considerations," "Data and Modeling Constraints," and "Organizational Governance."
  • Templates for question discussions, decision process guidelines, and success criteria were developed.

• Validation of Design Principles:

  • Confirmed that modular design supports flexible application across different institutional scenarios.
  • Ensured question design aligns with technical standards while addressing social and ethical dimensions.

Advantages Compared to Existing Solutions

• This toolkit is the first to be designed specifically from the perspective of public sector needs, addressing gaps in existing toolkits primarily tailored for industrial contexts.
• Introduces reflective discussions to balance technical metrics with societal impacts.
• Supports deep collaboration across departments, avoiding "siloed" design issues.

Experimental or Evaluation Results

• User Validation: Through co-design processes involving 32 participants, the applicability of the toolkit's questions and processes was confirmed.
• Participants found the toolkit effective in supporting early-stage decision discussions while prioritizing responsible AI development and social ethics.

Limitations and Future Directions

• Limitations:
  • Current design primarily targets internal institutional staff and does not directly include community member participation processes.
  • Long-term evaluation of the toolkit's practical impact has yet to be conducted.
• Future Directions:
  • Enhance the toolkit to support community member participation in specific AI decision discussions.
  • Explore cross-domain adaptation of the toolkit, such as its application in high-risk industrial AI contexts like healthcare.
  • Investigate policy incentives to promote widespread adoption of responsible AI toolkits.

Conclusion

The Situate AI Guidebook provides a structured, reflective, and collaborative decision support framework and toolkit for early-stage public sector AI design. Co-designed with public service institutions, frontline staff, and community advocates, the toolkit aims to prevent inappropriate development and use of AI tools in high-risk social contexts, offering a new template for advancing responsible AI toolkits.