"Should I Rely on You or the AI?" Leaders' Trust and Perceptions in Mixed Human-AI Teams

Paper Title

"Should I Rely on You or the AI? Leaders' Trust and Perceptions in Mixed Human-AI Teams"

Publication Info

• Topic area: Human-agent collaboration in hierarchical mixed teams.
• Keywords: Human-agent teams, trust, leadership, ability, integrity, hierarchical teams, mixed teams, human-AI collaboration, ABI framework, BW4T testbed.

Background and Problem

• Problem / challenge: Limited understanding of hierarchical mixed human-agent teams, particularly how human leaders manage and trust human versus agent followers based on ability and integrity.
• Significance: As autonomous agents increasingly collaborate with humans in industrial and organizational contexts, understanding trust dynamics is critical for effective coordination and adoption of human-agent teams.
• Motivation and related work: Prior research has focused on dyadic human-agent interactions or homogeneous teams, leaving gaps in understanding hierarchical mixed teams. The ABI framework has been applied to trust in automation, but its role in hierarchical mixed teams remains underexplored.

Solution

• Proposed approach: A lab study using the BW4T testbed to simulate hierarchical mixed human-agent teams, examining trust and perceptions of human and agent followers based on ability and integrity.
• Novelty:
  1. Empirical investigation of trust dynamics in hierarchical mixed human-agent teams.
  2. Identification of ability- and integrity-related themes shaping leaders' evaluations.
  3. Mixed-method analysis of leaders' preferences and qualitative reflections on human versus agent followers.
• Procedure and key techniques:
  • Teams of one human leader, one human follower, and two agent followers performed a block-moving task.
  • Agents were categorized into three types: High-Integrity-High-Ability (HI-HA), High-Integrity-Low-Ability (HI-LA), and Low-Integrity-High-Ability (LI-HA).
  • Trust, performance, preference, and qualitative comments were measured across six experimental conditions using surveys, task completion time, and thematic analysis.

Results

• Concrete findings:
  • Trust in human followers remained stable and high across conditions, while trust in agent followers varied significantly by type (HI-HA > HI-LA > LI-HA).
  • Leaders preferred human followers over agents, even when agents exhibited higher ability and integrity.
  • Team trust mirrored trust in agent followers and was highest under both-HI-HA conditions and lowest under both-LI-HA conditions.
  • Task completion time was shortest under both-HI-HA conditions and longest under both-HI-LA conditions.
• Advantage over baselines:
  • Human followers were preferred despite agents' superior memory and reliability, highlighting the importance of adaptability and perceived in-group similarity.
  • HI-LA agents were trusted more than LI-HA agents, emphasizing the critical role of integrity over ability in agent trust.
• Experiments / evaluation:
  • Participants: 72 individuals (36 leaders).
  • Measures: Trust ratings, task completion time, preference scores, thematic analysis of qualitative comments.
  • Manipulations: Agent ability (memory capacity) and integrity (command compliance).
• Limitations and future work:
  • Focused on leaders' perspectives; future work should include follower data.
  • Limited to a 2H-2A team structure; alternative compositions and co-leadership scenarios should be explored.
  • Used rule-based agents; future studies should examine more complex AI roles enabled by LLMs.
  • Short-term study; long-term effects of benevolence and social dynamics need investigation.

Summary

This study investigates trust and perceptions in hierarchical mixed human-agent teams, revealing distinct expectations for human versus agent followers. Human leaders consistently preferred human followers due to their adaptability and proactive behaviors, while trust in agents depended heavily on their integrity. The findings highlight the need for HAT designs that leverage humans' contextual flexibility and agents' controllability and precision. Future research should explore diverse team structures, long-term dynamics, and advanced AI roles to optimize mixed human-agent collaboration.