Trust in Collaborative Automation in High Stakes Software Engineering Work: A Case Study at NASA

Title of the Paper

Trust in Collaborative Automation in High Stakes Software Engineering Work: A Case Study at NASA

Paper Information

• Subject Areas: Human-Computer Interaction, Trust Modeling, Automated Software Engineering
• Keywords: Trust, Automation, Tools, Software Engineering, Human Behavior Studies, Case Study, NASA, High-Stakes Systems

Research Background and Issues

• Problems and Challenges:
  1. As the level of automation in software development tools increases, engineers' trust in these tools becomes critical, yet how to establish this trust remains a challenge.
  2. In application scenarios, inappropriate trust (over-trust or lack of trust) can lead to negative outcomes, such as wasted time on excessive verification or failure to identify critical issues.
  3. Existing research predominantly focuses on controlled laboratory environments, lacking studies on trust development mechanisms in real-world software engineering contexts.
• Significance of the Research:
  1. In high-stakes scenarios (e.g., space missions), engineers' distrust of automated tools can directly result in mission failure, incurring significant costs and reputational damage.
  2. Understanding how software engineers develop trust in tools can improve tool design, thereby enhancing efficiency and reliability.
• Research Motivation and Related Work:
  1. Current research on trust in automated systems primarily focuses on military or controlled environments, with little attention to the software engineering domain.
  2. Compared to existing work, this study focuses on the contextual factors influencing trust in the development of highly automated tools.

Solution

• Research Methodology: The authors propose a longitudinal, multi-method ethnographic study focusing on the interactions between engineers and automated tools in real-world scenarios (NASA). The qualitative methods employed include:

  • In-depth semi-structured interviews to analyze different dimensions of trust.
  • Think Aloud Studies to observe engineers' initial interactions with new tools.
  • Observations of participants' work practices and cultural contexts to identify potential factors influencing trust.

• Innovative Contributions:

  1. Introduced the concept of "collaborative trust," viewing trust as a dynamic collaboration between humans and automated tools rather than a static trust in a single system.
  2. Deconstructed the mechanisms of trust formation through a comprehensive framework incorporating transparency, usability, social/organizational factors, and procedural factors.

• Key Techniques and Implementation Steps:

  1. Over a 10-week period, combined interviews, observations, and user studies to identify 16 parameters influencing trust in automated tools.
  2. Used iterative thematic analysis to code and organize data, revealing trust development pathways from multiple perspectives.
  3. Provided detailed explanations of specific trust indicators such as "Mission Proven" and "Organizational Investment."

Research Findings

• Specific Outcomes:

  1. Identified four major dimensions influencing engineers' trust in software automation tools (transparency, usability, social factors, procedural factors) and their 16 specific manifestations:
     • Transparency: Visibility, feedback, documentation and training, long-term usage experience.
     • Usability: System complexity, bug transparency, intuitiveness.
     • Social Factors: Colleagues' usage experience and endorsements, developers' background and reputation.
     • Procedural Trust: System testing extent, mission validation, etc.
  2. Proposed a dynamic pathway for trust development in specific scenarios, emphasizing the social embeddedness of trust in environments where trial-and-error is limited.

• Comparative Advantages:

  1. Provided a novel framework distinct from traditional laboratory studies (research site: NASA's space mission center), uncovering intricate details of trust formation in real high-stakes scenarios.
  2. Introduced an innovative perspective that "trust is not in the tool itself but in the collaborative process," addressing a gap in current research on software engineering contexts.

• Experimental or Evaluation Results:

  1. Empirical findings indicate that tools with high transparency, mission validation, and organizational support are more likely to be trusted.
  2. During the adoption of new tools, complexity and "unknown bugs" emerged as significant barriers, highlighting the need to improve feedback mechanisms and simplify operational design.

• Limitations and Future Directions:

  1. The study's data, obtained through long-term field observations and interviews, has strong contextual dependency, so the results should be cautiously generalized to other domains.
  2. Future research is recommended to further validate the applicability of the "collaborative trust" model in other knowledge-intensive fields.
  3. Additionally, explore technical and design support elements to reduce complexity and enhance initial user trust.

Reference Value

This paper not only provides profound insights into trust modeling but also offers a practical reference framework for the design and evaluation of software engineering tools. It is particularly valuable for guiding tool design in high-stakes domains such as aerospace and medicine.