Debate Chatbots to Facilitate Critical Thinking on YouTube: Social Identity and Conversational Style Make A Difference

Title of the Paper

Using Debate Chatbots to Foster Critical Thinking on YouTube: The Importance of Social Identity and Conversational Style

Bibliographic Information

• Field of Study: Human-Computer Interaction (HCI), Artificial Intelligence Applications, Critical Thinking
• Keywords: Filter bubble, critical thinking, conversational agents, agent personality, online video platforms, language models, social computing, debate chatbots

Research Background and Problem

• Existing Issues:
  • Recommendation algorithms on video platforms like YouTube may create "filter bubbles," exposing users to similar viewpoints repeatedly, limiting access to diverse perspectives.
  • The immersive nature of long videos can lead users to passively accept information without engaging in critical analysis.
  • There is a lack of effective methods to encourage users to reassess their stance on video content after viewing.
• Importance of the Research:
  • Filter bubbles may exacerbate cognitive biases and social polarization.
  • Critical thinking is a key skill in education and information literacy, essential for combating filter bubbles.
• Research Motivation:
  • The development of large language models (LLMs) has enabled chatbots to act as debate partners in complex dialogues, but the impact of chatbot social identity and conversational style on user thinking remains unclear.
• Related Work:
  • Prior studies have explored AI systems to promote critical thinking, but few have focused on applications within social video platforms.
  • No empirical data exists to validate how chatbots address the filter bubble issue.

Proposed Solution

• Proposed Approach:
  • Develop and evaluate a debate chatbot based on large language models (LLMs) that engages users in debate after video viewing via a pop-up interface.
  • The chatbot initiates dialogue by presenting viewpoints opposing the user's stance, thereby fostering critical thinking.
• Innovative Aspects:
  • Introduce and evaluate two core personality variables of the chatbot: social identity (ingroup vs. outgroup) and rhetorical style (persuasive vs. confrontational).
  • Investigate how personality traits influence user critical thinking, engagement, and motivation.
• Implementation Steps and Techniques:
  • Experimental platform design: Simulate a YouTube video viewing environment where users watch videos aligned with their opinions, followed by interaction with the chatbot.
  • Mixed-method experimental design: Study with 36 participants, covering video topics such as "Online meetings are better than in-person meetings" and "Consumers should tip."
  • Data analysis: Use qualitative and quantitative methods to assess the chatbot's impact on users' critical thinking and stance changes.

Research Findings

• Specific Results:
  • Ingroup chatbots and persuasive rhetoric significantly enhanced users' critical thinking, particularly in explanation, analysis, and self-regulation abilities.
  • The combination of outgroup identity and persuasive rhetoric had the most notable effect on fostering self-awareness and reflection.
  • While the chatbot helped users reassess their arguments, its influence on actual stance changes was minimal.
• Comparison with Existing Solutions:
  • This study demonstrates the potential of AI debate systems to promote critical thinking on video platforms, highlighting the importance of personality design.
  • It reveals that designing social identity and rhetorical style effectively improves the quality of AI-human interaction.
• Experimental or Evaluation Results:
  • Participants' overall critical thinking score was 5.11 out of 7, with high emotional and cognitive engagement after interaction.
  • Videos reinforced users' initial stances, while the chatbot had limited impact on altering their positions.
• Limitations and Future Directions:
  • Limitations:
    • The video topics studied were relatively mild; generalizability to more sensitive or conflict-heavy topics remains to be tested.
    • LLM models may exhibit biases (e.g., political leanings, cultural blind spots).
    • The study was conducted as a short-term experiment; long-term validation in natural settings is needed.
  • Future Research Directions:
    • Explore how to optimize AI-generated arguments to better align with real-world contexts.
    • Further investigate the mechanisms by which other social identity variables (e.g., profession, age, religion) influence user acceptance and critical thinking.
    • Validate the system design's applicability on highly interactive but content-diverse platforms (e.g., TikTok, Twitter).

Output Format and Logical Analysis

• The structure and content aim to comprehensively cover all aspects of the research, including background issues, methodology, and findings.
• Avoid omitting critical details from experimental design, statistical results, or limitations, ensuring a thorough and accessible summary.