"That's another doom I haven't thought about": A User Study on AI Labels as a Safeguard Against Image-Based Misinformation

Paper Title

"That's another doom I haven't thought about": A User Study on AI Labels as a Safeguard Against Image-Based Misinformation

Publication Info

• Topic area: AI-generated content labeling and its impact on misinformation perception.
• Keywords: AI labeling, misinformation, generative AI, user trust, social media, image-based misinformation, AI-generated images, mislabeling, user study, transparency.

Background and Problem

• Problem / challenge: The effectiveness of AI-generated content labels in reducing misinformation is unclear, and mislabeling may lead to unintended consequences.
• Significance: AI-generated images are increasingly realistic and widespread, posing risks like misinformation, public manipulation, and erosion of trust in information sources.
• Motivation and related work: Previous studies focused on text-based AI labels or deepfake warnings but lacked comprehensive analysis of image-based AI labels, especially their side effects and user perceptions. This paper addresses these gaps.

Solution

• Proposed approach: A two-part study combining qualitative focus groups and a quantitative survey to assess user perceptions and the effects of AI labels on misinformation.
• Novelty:
  1. First exploration of user opinions and concerns about AI labels for images.
  2. Large-scale survey measuring the impact of AI labels on user judgments of true and false claims.
  3. Investigation of mislabeling consequences and their effect on user trust.
• Procedure and key techniques:
  • Focus groups: Five sessions with 18 participants from the U.S. and EU to gather qualitative insights on AI labeling expectations, concerns, and mechanisms.
  • Survey: Pre-registered online survey with 1,354 participants using a 2 × 2 × 3 factorial design to evaluate the effects of labeling and mislabeling on user accuracy, sensitivity, and response bias.
  • Analysis methods: Generalized linear mixed models (GLMMs), signal detection theory (SDT), and thematic coding.

Results

• Concrete findings:
  • AI labels reduced belief in false claims supported by AI-generated images (accuracy for false claims with labeled AIGIs: 84.9% vs. 82.1% without labels).
  • Labels caused unintended side effects: reduced belief in true claims illustrated with labeled AIGIs and increased susceptibility to false claims with human-made images.
  • Mislabeling (unlabeled AIGIs or wrongly labeled human-made images) further exacerbated these issues.
• Advantage over baselines: Labels improved accuracy for false claims with AI-generated images but failed to enhance overall claim verification or sensitivity.
• Experiments / evaluation:
  • Focus groups revealed user concerns about labeling mechanisms, mislabeling, and trust in platforms.
  • Survey tested labeling effects using realistic social media posts sourced from fact-checking sites.
  • Metrics included accuracy, sensitivity, response bias, and confidence levels.
• Limitations and future work:
  • Sample bias (tech-savvy participants from U.S. and EU).
  • Artificial survey setting with limited content types.
  • Future research should explore real-world interactions, framing effects, and evolving user trust in AI labels.

Summary

This study investigates the role of AI labels in combating image-based misinformation. While labels reduced belief in misleading AI-generated images, they introduced negative side effects, such as reduced trust in true claims with labeled AI images and increased susceptibility to false claims with human-made images. Mislabeling further eroded trust in labels. The findings highlight the need for careful design, transparency, and complementary measures to address misinformation effectively. Policymakers and platforms must consider user expectations and concerns to ensure AI labels are both effective and trustworthy.