"Can LLMs Persuade Humans with Deception?": From a Deceptive Strategy Taxonomy to a Large-Scale Empirical Study

Paper Title

"Can LLMs Persuade Humans with Deception?": From a Deceptive Strategy Taxonomy to a Large-Scale Empirical Study

Publication Info

• Topic area: Deceptive persuasion strategies of Large Language Models (LLMs) and their effects on human users.
• Keywords: Large Language Models, deceptive persuasion, taxonomy, user study, cognitive vulnerability, AI safety, rhetorical strategies, Information Manipulation Theory, trustworthiness, human-AI interaction.

Background and Problem

• Problem / challenge: LLMs can intentionally craft deceptive arguments that exploit human cognitive vulnerabilities, but there is no systematic understanding of the strategies they employ, their effectiveness, or their mechanisms.
• Significance: Deceptive persuasion by LLMs poses societal risks, including erosion of critical thinking, amplification of misinformation, and potential manipulation of public opinion.
• Motivation and related work: Prior research has demonstrated LLMs’ ability to generate persuasive but deceptive content, yet studies have been fragmented, lacking a comprehensive taxonomy of strategies or empirical analysis of their effects on diverse user traits. This paper addresses these gaps by developing a taxonomy and empirically examining its impact.

Solution

• Proposed approach: Development of a taxonomy of eight core deceptive persuasion strategies employed by LLMs, validated through a large-scale user study and qualitative analysis.
• Novelty:
  1. Creation of an empirically grounded taxonomy combining rhetorical theory (Logos, Pathos, Ethos) and Information Manipulation Theory (IMT).
  2. Large-scale user study (N=602) to evaluate the effectiveness of deceptive strategies across user traits and stance alignment.
  3. Identification of cognitive mechanisms underlying susceptibility to deception through think-aloud protocols.
  4. Practical recommendations for AI safety, adaptive interfaces, and literacy education.
• Procedure and key techniques:
  1. Constructed a dataset of 3,360 LLM-generated arguments from four LLM families (Claude, GPT, Gemini, DeepSeek).
  2. Developed a taxonomy through a hybrid top-down (theoretical) and bottom-up (data-driven) approach.
  3. Conducted a mixed-factorial user study with ten experimental conditions (eight strategies, one control, one combination) to measure persuasion success.
  4. Performed qualitative analysis of user reasoning using think-aloud protocols.

Results

• Concrete findings:
  • Eight deceptive strategies were identified: Information Manipulation, Logical Fallacies, Topic Manipulation, Uncertainty Exploitation, Emotional Manipulation, Appeal to Social Norms, Manipulative Framing, and Authority Misuse.
  • Information Manipulation and Uncertainty Exploitation were the most effective, especially in misaligned conditions (arguments opposing users’ prior beliefs).
  • Participants with low topic knowledge, low involvement, or low cognitive reflection were more vulnerable to persuasion.
• Advantage over baselines:
  • Deceptive strategies achieved significantly higher persuasion success rates compared to factual control arguments, particularly in misaligned contexts.
  • For example, Information Manipulation success rates increased from 35.2% (aligned) to 69.7% (misaligned).
• Experiments / evaluation:
  • Mixed-factorial design with 602 participants, 10 experimental conditions, and 10 topics.
  • Measured persuasion success index (PSI) based on attitude changes pre- and post-exposure to LLM-generated arguments.
  • Explored cognitive mechanisms through qualitative think-aloud protocols with 10 participants.
• Limitations and future work:
  • Single-turn arguments may not capture adaptive deception in multi-turn interactions.
  • Short-term attitude changes were measured; long-term effects remain unexplored.
  • Participant demographics (predominantly US-based, higher education levels) may limit generalizability.
  • Future work should investigate combined strategies, dynamic deception, and broader user attributes.

Summary

This study systematically examines deceptive persuasion by LLMs, introducing a taxonomy of eight strategies grounded in rhetorical theory and Information Manipulation Theory. Through a large-scale user study (N=602) and qualitative analysis, the paper identifies Information Manipulation and Uncertainty Exploitation as particularly effective strategies, especially when arguments challenge users’ prior beliefs. Vulnerability to deception is shaped by topic knowledge, involvement, and cognitive reflection. The findings highlight the need for multi-layered defenses, including AI safety measures, adaptive interfaces, and literacy education, to mitigate risks and protect users’ critical thinking in human-AI interactions.