Deceptive Explanations by Large Language Models Lead People to Change their Beliefs About Misinformation More Often than Honest Explanations

Research Background and Issues

• What problems or challenges did the authors identify?
  With the development of large language models (LLMs), the explanatory content they generate not only has the potential to spread misinformation but may also mislead the public through credible, logically coherent, and superficially truthful explanations. These "deceptive explanations" are more influential than simple misclassifications or honest explanations, particularly in cases involving fake news headlines and incorrect scientific explanations.

• Why is this issue important?
  This behavior could significantly impact the public's ability to distinguish between facts and misinformation, alter the credibility patterns of news media, and lead to the spread of misinformation in the public domain, thereby harming health, social governance, and public cognition. Furthermore, there is currently limited research on explanatory AI as a tool for deception, and in-depth analysis is lacking.

• Research Motivation and Related Work
  The motivation lies in uncovering the potential societal risks of AI-generated explanatory content, especially when explanations appear credible but are fundamentally incorrect. Related studies have shown that explanatory AI can enhance trust, but its potential as a misleading tool has been scarcely explored.

Solutions

• What methods or solutions did the authors propose?
  The study conducted online experiments to compare and evaluate the impact of deceptive AI-generated explanations, honest explanations, simple classifications, and classifications without explanations on belief changes. It also examined how logical validity and individual traits (e.g., cognitive reflection, trust in AI) moderated these effects.

• What are the innovative aspects of this solution?

  1. Introduced the concept of "deceptive explanations" and systematically assessed their amplifying effects on misinformation dissemination.
  2. Conducted detailed research on the role of individual factors (e.g., trust levels, logical reasoning ability) and logical validity in mitigating the misleading effects of AI.
  3. Utilized large-scale pre-registered online experiments to quantify the specific impact of different explanatory forms on human beliefs.

• What are the implementation steps and key technologies used?

  1. Experimental Design: Created a dataset of real and fake news headlines, generating both honest and deceptive explanations for each.
  2. Research Variables:
     • Explanation authenticity (honest vs. deceptive).
     • Logical validity of explanations (valid vs. invalid).
     • Individual moderating factors (e.g., cognitive reflection test scores, user confidence levels).
  3. Experimental Procedure: Participants evaluated the authenticity of each news headline, received AI-generated feedback (classification or classification with explanation), and adjusted their judgments accordingly.
  4. Data Analysis: Used linear regression models, significance testing, and other statistical tools to analyze relationships among variables.

Research Findings

• What specific findings were obtained?

  1. Deceptive explanations are more persuasive: Compared to honest explanations, deceptive AI-generated explanations significantly increased participants' trust in fake headlines and weakened their trust in genuine information.
  2. Importance of logical validity: Deceptive explanations with invalid logic were significantly less persuasive than those with valid logic, indicating that users can partially identify illogical content but remain susceptible to superficially credible explanations.
  3. Impact of individual factors: Cognitive reflection ability, trust in AI, and self-perceived knowledge levels did not significantly reduce the influence of deceptive explanations, especially when the content appeared credible.

• What advantages does it have compared to existing solutions?

  1. This study is the first to systematically examine the potential of explanatory AI in spreading misinformation, addressing a critical research gap.
  2. Through large-scale experiments and rigorous modeling, it quantified the effects of deceptive explanations, providing direct data support for designing more transparent and reliable AI systems in the future.

• What are the experimental or evaluation results?

  1. Experimental data showed that deceptive AI explanations were more impactful than honest explanations, significantly increasing participants' trust in misinformation.
  2. Under conditions of invalid logic, the persuasive power of deceptive explanations was significantly reduced, providing a basis for designing logical education and intervention strategies to counter misleading effects.

• Limitations and Future Directions
  Limitations:

  1. The AI model (GPT-3) used in the experiments has capability constraints; more advanced language models in the future may amplify these effects further.
  2. The experiments were conducted in controlled environments and did not fully simulate real-world social media or news consumption scenarios.

  Future Directions:

  1. Investigate how to develop AI interfaces that promote users' logical reasoning and critical thinking abilities.
  2. Test the combined effects of explanatory AI with other persuasive techniques, such as emotional appeals and authority.
  3. Explore longitudinal experiments to study long-term deception and belief formation changes.