The Impact of AI Trustworthiness Labels on the Perception of AI Products

Paper Title

The Impact of AI Trustworthiness Labels on the Perception of AI Products

Publication Info

• Topic area: The effect of trustworthiness labels on user perception and evaluation of AI products.
• Keywords: AI trustworthiness, trust calibration, user perception, acceptance, intention to use, graphical labels, trust in AI, AI literacy, brand evaluation, multi-level labels.

Background and Problem

• Problem / challenge: Potential users struggle to assess the trustworthiness of AI products due to a lack of accessible, transparent information. This leads to misplaced trust or distrust, hindering AI adoption and integration.
• Significance: Misplaced trust or distrust can obstruct AI acceptance, reduce its societal benefits, and lead to expectancy violations or reduced long-term adoption.
• Motivation and related work: Prior studies, such as those by Pfeuffer, demonstrated the potential of single-dimensional trustworthiness labels (focused on data security and privacy) to influence trust. However, these studies lacked ecological validity, did not explore multi-dimensional labels, and did not examine broader outcomes like acceptance or intention to use. This paper builds on these gaps.

Solution

• Proposed approach: Introduction of multi-level, graphical AI trustworthiness labels based on the European Commission's seven criteria for trustworthy AI. These labels indicate low, intermediate, or high trustworthiness.
• Novelty:
  1. Extends prior work to multi-dimensional trustworthiness labels reflecting global criteria.
  2. Assesses broader outcomes, including acceptance, intention to use, and brand evaluation.
  3. Employs a more ecologically valid setting with realistic product images and advertisements.
  4. Explores the moderating effects of individual trust in AI and AI literacy.
• Procedure and key techniques:
  • Participants evaluated hypothetical AI products (smart fridges, voice assistants) paired with trustworthiness labels and/or advertisements.
  • Trust, acceptance, intention to use, attributed value, and brand evaluation were measured.
  • Bayesian linear mixed models analyzed the effects of trustworthiness levels, advertisements, and individual differences (trust in AI, AI literacy).

Results

• Concrete findings:
  • Trust, acceptance, and intention to use increased with higher trustworthiness levels indicated by the labels.
  • Participants were willing to pay more for products with higher trustworthiness levels.
  • Baseline evaluations of unlabeled products corresponded to intermediate trustworthiness levels, indicating a bias.
  • Brand evaluations were unaffected by trustworthiness levels or advertisements.
  • Individual trust in AI and AI literacy amplified the effects of trustworthiness labels, especially for high and low trustworthiness levels.
• Advantage over baselines:
  • Trust, acceptance, and intention to use ratings were significantly higher for labeled products compared to unlabeled ones.
  • Attributed value scaled with trustworthiness level comparisons (low-intermediate, intermediate-high, low-high).
  • Labels effectively communicated trustworthiness even in a realistic advertisement setting.
• Experiments / evaluation:
  • Sample: 100 participants from Germany and Austria (mean age: 34.6 years, 71% male).
  • Design: Three phases—label evaluation, label comparison, and brand evaluation.
  • Metrics: Trust, acceptance, intention to use (Likert scales), attributed value (percent willingness to pay), and brand evaluation (9-point scale).
  • Tools: Bayesian linear mixed models with Jeffreys–Zellner–Siow priors.
• Limitations and future work:
  • Single exposure to labels may not be sufficient to affect brand evaluations; future studies should test multiple exposures.
  • The study focused on hypothetical products and EU-specific trustworthiness criteria; results may not generalize to real products or non-EU populations.
  • Further research is needed on the role of AI literacy in interpreting trustworthiness cues and on active users of AI systems.

Summary

This study demonstrates that multi-level, graphical AI trustworthiness labels effectively communicate trustworthiness and positively influence trust, acceptance, and intention to use AI products. Unlabeled products were perceived as having intermediate trustworthiness, highlighting a bias that could lead to misplaced trust or distrust. Individual trust in AI and AI literacy moderated the label effects, suggesting the need for both trustworthiness labels and efforts to improve AI literacy. These findings support the implementation of trustworthiness labels as a means to enhance user decision-making and promote the responsible adoption of AI.