Beyond Decision Making: Considering Collaboration and Agency in the Design of AI-Based Decision-Support Systems for Fast-Response Medical Teams

In addition to aiding decision making, AI-based clinical decision-support systems may need to consider and support provider agency and collaboration between medical providers. We analyzed collaboration and agency within fast-response teams, identifying implications for designing decision-support systems that not only facilitate decision making, but also collaboration and agency. Using an Actor-Network Theory approach, we reviewed videos of 12 pediatric trauma resuscitations and conducted a secondary analysis of 27 interviews with trauma team members. We identified actants in trauma resuscitation, shifts in agency that can occur within fast-response teams during medical emergencies, and factors considered by providers when envisioning the design of decision-support systems. From our analysis, we propose implications for existing human-AI guidelines when designing AI systems for fast-response medical teams. We also highlight parallels between the introduction of clinical practice guidelines and the introduction of AI-based decision-support systems, suggesting that these systems may influence the training and "clinical gaze" of providers.