How Much Trust is Enough? Towards Calibrating Trust in Technology

Paper Title

How Much Trust is Enough? Towards Calibrating Trust in Technology

Publication Info

• Topic area: Trust calibration in Human-Computer Interaction (HCI) and measurement tools.
• Keywords: Trust calibration, Human-Computer Interaction, Human-Computer Trust Scale, trust propensity, trust interpretation, autonomous systems, psychometric tools, trust measurement, trust constructs, trust evaluation.

Background and Problem

• Problem / challenge: Existing tools for measuring trust propensity, such as the Human-Computer Trust Scale (HCTS), lack generalizable interpretation guidelines, making it difficult to assess trust levels consistently across contexts.
• Significance: Miscalibrated trust in technology can lead to overtrust (misuse) or undertrust (disuse), which may result in inefficient interactions or risks in sensitive domains like healthcare and law enforcement.
• Motivation and related work: Previous research has focused on fostering trust, but there is a growing need to calibrate trust to align with a system's actual capabilities. While the HCTS is a validated tool, its context-dependent results and lack of general interpretation guidelines limit its broader applicability.

Solution

• Proposed approach: Development of evidence-based interpretation guidelines for the HCTS, including thresholds for undertrust, adequate trust, and overtrust, inspired by the System Usability Scale (SUS) methodology.
• Novelty:
  1. Introduction of an adjective scale to complement HCTS results and facilitate interpretation.
  2. Establishment of empirically derived thresholds for interpreting HCTS scores across contexts.
  3. Proposal of a practical framework for trust calibration, emphasizing moderate trust as optimal.
  4. Validation of the approach through two empirical studies on distinct technologies.
• Procedure and key techniques:
  1. Conducted two studies using HCTS to assess trust propensity for Facial Recognition Systems (Study 1, N=711) and Biometric Payment Systems (Study 2, N=227).
  2. Introduced a 7-point adjective scale for participants to describe their trust levels.
  3. Used ANOVA and post-hoc tests to identify statistically significant differences between trust categories.
  4. Calculated cut-offs for HCTS scores based on confidence intervals and merged adjacent categories when necessary.
  5. Proposed generalizable thresholds for interpreting HCTS results and visualized them as a continuum.

Results

• Concrete findings:
  • Derived thresholds for interpreting HCTS scores: undertrust (≤2.30), adequate trust (2.31–3.60), and overtrust (≥3.61).
  • Cronbach’s Alphas for HCTS items were 0.814 (Study 1) and 0.860 (Study 2), confirming reliability.
  • Final recoded adjective scale categories showed statistically significant differences in both studies.
• Advantage over baselines:
  • Provides a structured, empirically grounded framework for interpreting HCTS results, addressing the tool's prior limitations.
  • Facilitates trust calibration by identifying undertrust and overtrust, enabling actionable insights for system design and evaluation.
• Experiments / evaluation:
  • Studies conducted via online surveys using video stimuli to simulate interactions with two distinct technologies.
  • Participants rated trust using HCTS and the adjective scale; data analyzed to establish thresholds and validate the interpretation framework.
• Limitations and future work:
  • Limited to two technologies with similar technical characteristics, reducing generalizability.
  • Contextual and subjective nature of trust measurement requires further validation across diverse domains.
  • Future research should refine thresholds, explore trust constructs separately, and investigate the interplay between trust dimensions.

Summary

This paper addresses the challenge of interpreting trust propensity in technology by developing empirically grounded guidelines for the Human-Computer Trust Scale (HCTS). Through two studies on distinct technologies, the authors propose thresholds for undertrust, adequate trust, and overtrust, emphasizing the importance of trust calibration. The findings highlight the HCTS as a reliable tool for assessing trust propensity, with practical implications for researchers and practitioners in designing and evaluating autonomous systems. Future work is needed to validate the approach across broader contexts and refine the interpretation framework further.