Empowering Calibrated (Dis-)Trust in Conversational Agents: A User Study on the Persuasive Power of Limitation Disclaimers vs. Authoritative Style

Document Title

Empowering Calibrated (Dis-)Trust in Conversational Agents: A User Study on the Persuasive Power of Limitation Disclaimers vs. Authoritative Style

Document Information

• Topic Area: Trust and persuasion techniques in human-computer interaction
• Keywords: Large language models, conversational agents, chatbots, ChatGPT, automated trust, elaboration likelihood model, communication style

Research Background and Issues

• Issues and Challenges

  • Conversational agents based on large language models (LLMs) have driven progress in many domains but are prone to generating erroneous information.
  • Lay users may struggle to identify misinformation during interactions with these systems, leading to overtrust and further dissemination of incorrect information.
  • The primary challenge lies in helping users accurately assess the reliability of information provided by conversational agents and avoiding excessive trust.

• Importance

  • This issue pertains not only to effective system design but also to user safety and ethical considerations.
  • Understanding how users develop trust in conversational agents and how such trust influences their attitudes and behaviors contributes to the design of safer and more effective AI systems.

• Research Motivation and Related Work

  • Previous studies have focused on users' perceptions of misinformation but rarely explored how modifying system design and style can adjust user trust.
  • The elaboration likelihood model (ELM) is considered a crucial theoretical framework for analyzing trust and attitude formation.

Solution

• Methods and Solutions

  • Propose an experimental design to test the impact of two key factors on user trust through an online user study:
    1. Preliminary Information: Warnings about potential errors from the agent or prompts highlighting its positive functionalities.
    2. Communication Style: Whether the agent's expressions during interaction are authoritative.

• Innovations

  • Treat user trust as a result of attitude change, integrating ELM theory to clarify the role of communication style and preliminary information as trust cues.
  • Define conversational agents as dual-role entities (trust object and information sender) to analyze their impact on persuasion and attitude.

• Implementation Steps and Techniques

  • Conduct an online study simulating two conversations: one to showcase different communication styles, and another to provide persuasive information.
  • Set up a 2×2 experiment to manipulate preliminary information (positive functionality vs. limitations) and communication style (high authority vs. low authority).
  • Collect quantitative data on user trust, attitude changes, and related moderating factors (e.g., need for cognition and self-efficacy in automation technology).

Research Findings

• Specific Findings

  • Communication style (high authority vs. low authority) significantly impacts user trust and attitude formation, while warning information fails to effectively alter user trust.
  • A high-authority communication style leads to greater user trust, which enhances the persuasive effect of the conversational agent's information.
  • There is a significant relationship between users' attitudes toward robots in public spaces and their trust in conversational agents.

• Advantages

  • Provides a critical perspective on current conversational agent design: textual warnings are limited, while communication style plays a more significant role.
  • Offers specific recommendations for designing more human-centric conversational agents, such as dynamically adjusting user trust through communication style rather than relying solely on fixed warnings.

• Experimental and Evaluation Results

  • The experiment shows that in the absence of active processing of preliminary warning information, users are more easily persuaded by authoritative styles.
  • High-authority styles enhance the consistency of user attitudes, particularly when users have a lower need for cognition.

• Limitations and Future Directions

  • The experiment did not directly simulate scenarios where agents generate erroneous information, which may limit insights into actual user behavior.
  • Future research should explore how users' exploratory behaviors during real interactions with agents influence trust formation.
  • Advocate for further testing of real-time uncertainty prompts in application design and studying the effectiveness of dynamically adjusting trust through personalized styles.

Conclusion

This study highlights trust issues in current LLM-based conversational agent design and demonstrates that more flexible or non-authoritative communication styles can more effectively calibrate user trust. The findings provide valuable references for developers, designers, and researchers in related fields, supporting the design of safer, more effective, and ethically compliant AI systems.