Slip Through the Chat: Subtle Injection of False Information in LLM Chatbot Conversation Increases False Memory Formation
Authors
This study examines the potential for malicious generative chatbots to induce false memories by injecting subtle misinformation during user interactions. An experiment involving 180 participants explored five intervention conditions following the presentation of an article: (1) no intervention, (2) reading an honest or (3) misleading article summary, (4) discussing the article with an honest or (5) misleading chatbot. Results revealed that while the misleading summary condition increased false memory occurrence, misleading chatbot interactions led to significantly higher rates of false recollection. These findings highlight the emerging risks associated with conversational AI as it becomes more prevalent. The paper concludes by discussing implications and proposing future research directions to address this concerning phenomenon.
Research Questions / Practical Problems
Question signals indexed for this paper.
Research Questions
3- Can generative large language model-driven chatbots induce users to form false memories by injecting misleading information?Category: Conversational Agent and Chatbot DesignSimilar questionsarrow_forward
- Is interactive misleading chat more likely to induce false memories in users than static summaries?Category: Conversational Agent and Chatbot DesignSimilar questionsarrow_forward
- Can false memories be consistently formed across different topic domains through chatbots?Category: Conversational Agent and Chatbot DesignSimilar questionsarrow_forward
Practical Problems
1- Users are easily misled when chatting with AI and remember incorrect information.Category: Conversational Agent and Chatbot DesignSimilar questionsarrow_forward
- 60%
A Canary in the AI Coal Mine: American Jews May Be Disproportionately Harmed by Intellectual Property Dispossession in Large Language Model Training
CHI '24· AI Ethics, Fairness & Accountability +1
- 60%
Unpacking Trust Dynamics in the LLM Supply Chain: An Empirical Exploration to Foster Trustworthy LLM Production & Use
CHI '25· AI Ethics, Fairness & Accountability +1
- 60%
Deceptive Explanations by Large Language Models Lead People to Change their Beliefs About Misinformation More Often than Honest Explanations
CHI '25· AI Ethics, Fairness & Accountability +1
- 60%
AI Mismatches: Identifying Potential Algorithmic Harms Before AI Development
CHI '25· AI Ethics, Fairness & Accountability +1
- 60%
Artificial Intelligence and Risk in Design
DIS '20· AI Ethics, Fairness & Accountability +1
Based on Jaccard similarity of research subtopics & professions (≥60%)