Oops, I Did It Again (But I Know It): Robot Failure Consistency and Awareness in Human-Robot Collaboration

Paper Title

Oops, I Did It Again (But I Know It): Robot Failure Consistency and Awareness in Human-Robot Collaboration

Publication Info

• Topic area: Human-robot interaction, focusing on failure patterns, trust, and perceived intelligence.
• Keywords: Human-robot collaboration, robot failures, trust repair, failure awareness, perceived intelligence, failure severity, homogeneous vs. heterogeneous failures, trust dynamics, expectancy violation, collaborative robotics.

Background and Problem

• Problem / challenge: Repeated robot failures can erode trust and perceptions of intelligence, but the effects of failure sequence, severity, and robot awareness remain poorly understood.
• Significance: Understanding these dynamics is critical for designing robots that maintain user trust and collaboration effectiveness, even in the face of inevitable errors.
• Motivation and related work: Prior research has explored single failures, trust repair strategies, and failure severity, but gaps remain in understanding how failure sequence and robot awareness interact to shape user perceptions over time. This study builds on Expectancy Violation Theory (EVT) and trust calibration models to address these gaps.

Solution

• Proposed approach: A controlled study investigating the effects of failure sequence (homogeneous vs. heterogeneous), failure severity, and robot awareness (none, partial, full) on trust and perceived intelligence during a collaborative Tangram puzzle task.
• Novelty:
  1. Analysis of how homogeneous vs. heterogeneous failure sequences influence trust and perceived intelligence.
  2. Examination of robot awareness levels (none, partial, full) across different failure types (freezing, planning, grasping).
  3. Investigation of how failure severity interacts with prior failures to shape user perceptions.
• Procedure and key techniques:
  • Participants (N=54) collaborated with a robot on six Tangram puzzles, encountering three pre-programmed failures.
  • Failures included freezing (temporary pause), planning (incorrect placement), and grasping (failed object pickup).
  • Robot awareness was manipulated across three levels: no awareness, partial awareness (acknowledgment without correction), and full awareness (acknowledgment with correction).
  • Trust and perceived intelligence were measured using validated scales, and qualitative feedback was collected through open-ended questions.

Results

• Concrete findings:
  • Heterogeneous failure sequences led to greater reductions in perceived intelligence compared to homogeneous sequences, particularly after the second failure.
  • Trust and perceived intelligence were higher when the robot demonstrated awareness (partial or full) for grasping and planning failures, but awareness had no effect for freezing failures.
  • Severe failures following mild ones caused sharper trust declines, consistent with Expectancy Violation Theory.
• Advantage over baselines:
  • Robots with partial or full awareness outperformed those with no awareness in maintaining trust and perceived intelligence for noticeable errors (grasping and planning).
  • Homogeneous failure sequences were less detrimental to perceived intelligence than heterogeneous sequences.
• Experiments / evaluation:
  • Mixed-design study with 54 participants, using a collaborative Tangram puzzle task.
  • Quantitative analysis with cumulative link mixed models (CLMMs) and qualitative thematic analysis of participant feedback.
  • Measures included trust (7-point Likert scale) and perceived intelligence (5-point Likert scale).
• Limitations and future work:
  • Controlled laboratory setting may not generalize to real-world scenarios with higher complexity and safety stakes.
  • Limited failure types and pre-scripted awareness behaviors restrict ecological validity.
  • Future work should explore adaptive, real-time trust repair strategies, longer-term interactions, and multi-agent contexts.

Summary

This study investigates how failure sequence, severity, and robot awareness affect trust and perceived intelligence in human-robot collaboration. Results show that heterogeneous failure sequences and severe failures following mild ones erode trust and perceived intelligence more than homogeneous sequences or consistent severity. Robot awareness improves user perceptions for noticeable errors but has no effect for subtle ones like freezing. These findings highlight the importance of sequence-aware and context-sensitive trust repair strategies, where robots adapt their responses based on failure type, severity, and sequence to maintain effective collaboration.