When the Codec Hallucinates: User Perceptions of Miscompressed Images

Paper Title

When the Codec Hallucinates: User Perceptions of Miscompressed Images

Publication Info

• Topic area: User perception of neural image compression artifacts and their social implications.
• Keywords: Neural image compression, miscompression, user perception, semantic integrity, image artifacts, misunderstandings, intentional editing, distortion, trust in images, human-computer interaction.

Background and Problem

• Problem / challenge: Neural image compression methods can introduce subtle but semantically significant changes to image details, termed miscompressions. These changes lack visible indicators and may be mistaken for intentional edits, potentially leading to misunderstandings and eroding trust in images.
• Significance: With neural compression poised for widespread adoption, understanding its social and perceptual implications is critical to mitigate risks such as misinformation, false accusations, and errors in critical domains like surveillance and autonomous driving.
• Motivation and related work: While prior research has explored technical aspects of neural compression and user perception of image quality, no studies have empirically investigated how users perceive miscompressions or their social consequences. This paper addresses this gap by examining user perceptions of miscompressions compared to conventional compression artifacts.

Solution

• Proposed approach: A user study involving 115 participants who compared original images with conventionally compressed, neurally compressed, and miscompressed images to assess perceptions of misunderstandings, intentional editing, and uncontrollable distortion.
• Novelty:
  1. Introduced the first empirical method to study user perceptions of miscompressions.
  2. Provided evidence that miscompressions are perceived differently from conventional artifacts, increasing risks of misunderstandings and being mistaken for intentional edits.
  3. Derived implications for interface designs to mitigate risks associated with miscompressions.
• Procedure and key techniques:
  • Conducted a full-reference study in a controlled lab environment.
  • Participants viewed curated image pairs and rated differences on a 6-point scale for misunderstanding risk, intentional editing, and uncontrollable distortion.
  • Used a mixed between-subjects and within-subjects design, with 1,380 image views and 1,131 ratings collected.
  • Applied linear regression with fixed effects to analyze responses.

Results

• Concrete findings:
  • Miscompressions significantly increased perceived risks of misunderstandings (effect size d = 0.86).
  • Differences in miscompressed images were more likely attributed to intentional editing (d = 0.78) and less likely to uncontrollable distortion (d = 0.64).
  • Participants recognized JPEG artifacts as distortions but not miscompressions, which were often mistaken for edits.
• Advantage over baselines: Miscompressions were perceived as more likely to cause misunderstandings and resemble intentional edits compared to conventional JPEG artifacts, which were seen as distortions.
• Experiments / evaluation:
  • Stimuli included 36 image pairs (miscompressed, neurally compressed, JPEG, and uncompressed) selected for diversity in content and semantic relevance.
  • Participants were asked to identify differences and rate their effects and causes.
  • Results supported all three hypotheses with statistical significance (p < 0.001).
• Limitations and future work:
  • Sample limited to German-speaking computer science students, not representative of the general population.
  • Stimuli were hand-selected, introducing potential bias.
  • Future research should involve diverse populations, high-stakes scenarios, and expert evaluations to generalize findings.

Summary

This study empirically demonstrates that neural image compression can introduce miscompressions—subtle but semantically significant changes that users often mistake for intentional edits. Miscompressions increase the risk of misunderstandings and are not recognized as compression artifacts, unlike conventional JPEG distortions. The findings highlight the need for awareness and interface designs to mitigate these risks, especially as neural compression becomes more widely adopted. Future research should explore broader contexts and user populations to further understand and address the implications of miscompressions.