Lies, Deceit, and Hallucinations: Player Perception and Expectations Regarding Trust and Deception in Games

Title of the Paper

Lies, Deceit, and Hallucinations: Player Perception and Expectations Regarding Trust and Deception in Games

Bibliographic Information

• Subject Areas: Human-Computer Interaction, Video Game Design, Natural Language Generation, and User Experience
• Keywords: Video games, NPC dialogue, trust, LLM hallucinations, player experience, narrative complexity, game design, virtual agents, intention attribution

Research Background and Issues

• Identified Problems or Challenges:

  • In video games, the behavior and dialogue of NPCs (non-player characters) play a crucial role in building player trust and enhancing the gaming experience. However, there is relatively little research on how players perceive lies or false statements made by NPCs.
  • Recently, the use of large language models (LLMs) to generate NPC dialogue has become a trend. However, LLMs often "hallucinate," producing factually incorrect content that lacks explicit developer intent.

• Significance:

  • Lies, false statements, and deception can enhance narrative complexity, making game content deeper and more engaging. However, unpredictable false content generated by AI may undermine players' trust in the game, thereby affecting the overall experience.

• Research Motivation and Related Work:

  • Existing research on NPC credibility and the perception of lies primarily focuses on multiplayer online games or specific economic and social theories. There is a lack of exploration into how players perceive the authenticity of NPCs in single-player games and how this impacts game design.
  • This study combines intentionally designed false content with erroneous content generated by LLMs to analyze how players perceive, attribute, and react to these deceptive elements.

Solution

• Proposed Methods or Solutions:

  • Developed a 2D role-playing game called "AlphaBetaCity" to study players' perceptions of false statements made by NPCs.
  • The game includes both "intentional" false statements designed by humans and "unintentional" erroneous content generated by LLMs, allowing for a comparison of players' attitudes toward these two types of false information.
  • The game design features three task scenarios (familiarization task, search task, information task), where specific NPCs mislead players to incorrect locations or provide inaccurate information.

• Innovations:

  1. Introduced LLM-generated dialogue into single-player games and conducted experimental analysis combining "intentional" and "unintentional" false statements.
  2. Explored the narrative impact of hallucinated content generated by LLMs, particularly how such content, when devoid of narrative intent, elicits cognitive and emotional reactions from players.
  3. Engaged in interdisciplinary discussions involving psychology, philosophy, and game design, offering new perspectives on trust modeling and narrative artistry.

• Implementation Steps and Key Technologies:

  1. Game Development: Designed a traditional 2D RPG-style game featuring game scenarios and dialogue systems with specific false statements, incorporating LLMs (ChatGPT 3.5) to generate dialogue scripts.
  2. User Research: Recruited 29 participants to play the game and participate in semi-structured interviews to capture authentic player impressions of NPC interactions.
  3. Data Analysis: Employed qualitative thematic analysis, combining coding and inductive summarization to extract key themes related to trust-building and attribution of false statements.

Research Findings

• Specific Findings:

  • Players initially trust NPCs at the start of the game, but this trust may gradually erode due to false statements.
  • Players' attribution of false statements is influenced by several factors, including the consistency of NPC roles and behavior, the verifiability of statements, the consequences of the statements, and the perception of NPCs' "intent to deceive."
  • When players recognize that false content is generated by LLMs and lacks intentionality, their trust in the game and NPCs significantly decreases, and the depth of interaction is diminished.

• Comparison with Existing Solutions and Advantages:

  • This study explicitly distinguishes between two forms of false content (intentionally designed and unintentionally generated) and is the first to quantitatively analyze their differences in user experience.
  • By addressing the limitations of LLM-generated text (especially hallucination issues) within the dynamic narrative context of video games, the study deepens the social and technical implications of such technologies.

• Experimental and Evaluation Results:

  • False statements that align with the narrative framework or have consequential outcomes are perceived as acceptable and can even enhance immersion. In contrast, unintentionally false content generated by AI, due to its lack of "meaning," makes players feel disconnected.
  • The majority of players reported that knowing LLMs were involved in dialogue generation reduced their trust in NPCs. Additionally, players expected false statements to eventually be justified or resolved through the storyline.

• Limitations and Future Directions:

  • The study used a simplified experimental game, which does not fully cover all types of games (e.g., AAA games or other narrative styles).
  • The game primarily targeted young players with higher educational backgrounds and familiarity with technology, lacking broader demographic representation.
  • Future research should develop cross-genre, more complex test cases and delve deeper into optimizing the quality and gameplay compatibility of LLM-generated text.

This study, through the development of a specific experimental game and the integration of LLM-generated content, comprehensively analyzes the potential applications of false statements in game design and player experience, offering both theoretical and practical value.