Is this AI trained on Credible Data? The Effects of Labeling Quality and Performance Bias on User Trust

Title of the Paper

Is this AI trained on Credible Data? The Effects of Labeling Quality and Performance Bias on User Trust

Paper Information

• Field of Study: Research on the relationship between AI credibility and user trust
• Keywords: Training data credibility, labeling quality, labeling source, AI trust, algorithm bias

Research Background and Problem

• Identified Issues or Challenges: Current artificial intelligence (AI) systems lack transparency and credibility, particularly regarding issues of racial bias. Users' perceptions of the credibility of training data and AI bias may influence their trust, but the exact relationship remains unclear.
• Significance: As AI is increasingly applied in critical domains (e.g., healthcare, judiciary), understanding how to effectively enhance user trust in AI is crucial for promoting fairness and societal acceptance of AI.
• Research Motivation and Related Work:
  • The study builds on discussions in "algorithm bias" and "explainable AI," aiming to explore the impact of training data transparency and labeling information on user trust.
  • Previous research has suggested that showing training data statistics or explaining racial backgrounds can increase user trust, but systematic experimental validation is lacking.

Solution

• Proposed Solution:
  • A user perception model is proposed to explore how labeling quality, labeling source, and AI performance bias influence perceptions of training data credibility, thereby affecting user trust.
• Innovations:
  • Introduced the concept of "training data credibility" as a mediating variable for the first time to study the formation of human trust in AI.
  • Demonstrated a method to enhance transparency through "snapshots of labeling accuracy," avoiding the limitations of traditional post-hoc explanations.
• Implementation Steps and Technical Methods:
  • Designed an experiment with a 2 (labeling quality: high vs. low) × 4 (labeling source: third-party labeling vs. user-prompted labeling vs. voluntary user labeling vs. mandatory user labeling) × 3 (AI performance: no performance vs. unbiased performance vs. racially biased performance) factorial design.
  • Conducted a user study (N=430) to validate how labeling quality, labeling source, and racial bias influence perceptions of training data credibility and trust (cognitive trust, emotional trust, and behavioral trust).

Research Findings

• Specific Findings:
  1. High-quality labeling leads to higher perceptions of training data credibility.
  2. Perceptions of training data credibility positively influence users' cognitive trust and behavioral trust but have limited impact on emotional trust.
  3. Racial bias in AI (e.g., differences in classification accuracy for White or Black individuals) significantly weakens the positive effect of training data credibility on cognitive trust.
  4. Labeling source (e.g., self-labeling vs. third-party labeling) has a weaker impact on evaluations of training data credibility.
• Comparison with Existing Solutions:
  • Offers a "proactive transparency" design approach, differing from traditional post-hoc explanation methods in AI.
  • Emphasizes the visualization of training data and labeling quality, in addition to performance metrics, as key to enhancing user trust.
• Experimental or Evaluation Results:
  • Users are more inclined to trust AI with high-quality labeling, but in the presence of racial bias, cognitive trust is undermined even if the labeled data is credible.
  • AI performance bias has a greater impact on cognitive trust than training data credibility.
• Limitations and Future Directions:
  1. The experiments in this study primarily focus on visual data scenarios; further validation is needed for other domains, such as speech or language models.
  2. High-quality labeling is defined as 100% accuracy; future research could explore the minimum threshold for credibility (e.g., 80%-90% accuracy).
  3. Users may react negatively to excessive information, necessitating exploration of the balance between transparency and user experience.

Conclusion

This study demonstrates that displaying labeling quality is an effective way to enhance user trust in AI. However, AI bias severely undermines trust, highlighting the need for user-centered design to prevent over-trust issues. The research provides a new theoretical model and practical guidance for advancing AI's social responsibility and transparency.