The Limits of Stakeholder Participation in Safety-Critical Contexts: Lessons from Air Traffic Control

Paper Title

The Limits of Stakeholder Participation in Safety-Critical Contexts: Lessons from Air Traffic Control

Publication Info

• Topic area: Stakeholder participation in safety-critical system development.
• Keywords: Stakeholder participation, safety-critical systems, Air Traffic Control, responsible AI, user involvement, decision agency, transparency, participatory design, automation, socio-technical systems.

Background and Problem

• Problem / challenge: Current guidance on stakeholder involvement often advocates for high levels of decision agency across contexts without accounting for constraints in safety-critical domains. This can lead to tensions between user needs and non-negotiable requirements like safety and regulatory compliance.
• Significance: Understanding these constraints is crucial for designing systems that balance stakeholder input with operational safety and regulatory demands, particularly in domains like Air Traffic Control (ATC) where errors can have catastrophic consequences.
• Motivation and related work: Traditional participatory design literature highlights the benefits of stakeholder involvement but often relegates users to passive roles. Responsible AI literature promotes high stakeholder agency but lacks context-specific adaptations. This study builds on these gaps by empirically exploring stakeholder involvement in ATC systems development.

Solution

• Proposed approach: A case study of stakeholder involvement in ATC systems development, focusing on the experiences of Air Traffic Control Operators (ATCOs) and other employees.
• Novelty:
  1. Demonstrates how competing demands in safety-critical domains constrain user agency, adding nuance to responsible AI discussions.
  2. Highlights the role of transparency in improving stakeholder acceptance of constrained agency.
  3. Provides practical implications for adapting participatory practices in safety-critical and other domains.
• Procedure and key techniques:
  • Conducted semi-structured interviews with 11 ATCOs to understand current involvement practices, motivations, and dissatisfaction.
  • Held an 8-hour focus group with 11 participants, including ATCOs and other employees, to explore contextual constraints and potential solutions.
  • Thematic analysis of qualitative data to identify key themes and insights.

Results

• Concrete findings:
  • ATCOs are often involved late in the development process, limiting their impact on high-level decisions.
  • Safety-critical constraints, regulatory demands, and financial limitations restrict the flexibility to incorporate user feedback.
  • Transparency in decision-making processes can improve stakeholder satisfaction and perceived legitimacy.
• Advantage over baselines: Adds empirical evidence to the discourse on stakeholder involvement, challenging the universal applicability of high agency recommendations and emphasizing the need for context-specific adaptations.
• Experiments / evaluation:
  • Interviews revealed dissatisfaction with limited impact and non-transparent decision processes.
  • Focus group discussions identified contextual constraints and proposed earlier involvement and structured feedback mechanisms.
• Limitations and future work:
  • Small sample size due to the specialized nature of the domain.
  • Findings are based on a single case study; further research in other safety-critical and non-safety-critical domains is needed.

Summary

This study investigates stakeholder involvement in safety-critical contexts through a case study of ATC systems development. It finds that competing demands such as safety, regulatory compliance, and financial constraints limit user agency, challenging the universal applicability of high agency recommendations. Transparency in decision-making processes is highlighted as a key factor in improving stakeholder satisfaction and legitimacy. The findings have implications for adapting participatory practices in both safety-critical and other domains, emphasizing the need to balance stakeholder input with contextual constraints. Future research should explore these dynamics in other domains and develop tools to support effective stakeholder engagement.