"Ronaldo's a poser!": How the Use of Generative AI Shapes Debates in Online Forums

Research Background and Issues

• What problems or challenges did the authors identify?

  1. Online forum debates have the potential to foster critical thinking, but they can also generate discord and hostility.
  2. With the widespread adoption of Generative AI (GenAI), its use in online debates may introduce new issues, such as information bias, the spread of misleading content, and potential negative impacts on the online discussion ecosystem.

• Why is this issue important?

  1. Online debates are a significant form of idea exchange on social media, but they are often confrontational and may exacerbate emotional conflicts.
  2. While GenAI might improve debate efficiency, balancing its contributions to information accuracy and diversity is particularly urgent.

• Research Motivation and Related Work

  1. Current research primarily focuses on the impact of GenAI on individual writing tasks, with limited exploration of its use in online debate scenarios.
  2. Previous studies have employed traditional argumentation models, such as the Toulmin model, to examine the logical structure in persuasion models. However, the specific behavioral patterns and long-term effects of GenAI in multi-user debate interactions warrant further investigation.

Solutions

• What methods or solutions did the authors propose?

  1. The authors designed a small online forum centered on the football topic "Messi vs. Ronaldo: Who is better?" where users debated with the support of the generative AI tool ChatGPT.
  2. The study was divided into two phases: the first phase involved a turn-based debate between two participants, while the second phase introduced a third party for free-form debate.

• What are the innovative aspects of this solution?

  1. The solution combines the adversarial and collaborative nature of online debates with the collaborative writing mechanisms of generative AI, exploring how GenAI functions in scenarios involving multiple simultaneous users.
  2. By integrating content analysis and thematic analysis, the study examined forum posts, ChatGPT logs, and interview data to comprehensively explore user interactions with GenAI.

• What are the implementation steps and key technologies used?

  1. A small online debate forum was built on the "Forumotion" platform, restricting users to ChatGPT as their sole external information source.
  2. Data collection included forum posts, ChatGPT usage logs, and subsequent semi-structured interview content.
  3. Qualitative content analysis and thematic analysis were employed to decode users' argument structures, AI usage patterns, and user experience behaviors.

Research Findings

• What specific findings were achieved?

  1. Usage Patterns:

     • Participants tailored ChatGPT's responses to better fit the debate context by setting specific contexts or roles (e.g., "professional debater" or "Messi fan") and even prompted it to generate more aggressive content.
     • Many generated posts exhibited logical fallacies, such as hasty generalizations, straw man arguments, and ad hominem attacks.
     • When new debaters joined, many participants reduced their reliance on ChatGPT and focused more on interacting with other users.

  2. Debate Structure:

     • User posts contained three main argumentative elements: claims (including value claims, descriptive claims, etc.), evidence (statistical data, personal observations, examples, etc.), and reasoning (including rebuttals, analogies, and logical fallacies).
     • Value claims and evidence based on examples were the most frequently used, while hasty generalizations were the most common logical fallacy.

  3. Dynamic Changes:

     • The introduction of a third party encouraged more creative interactions among existing users, suggesting that fresh perspectives can stimulate deeper group thinking. However, GenAI was perceived more as a supporting tool rather than a leading role.

• What advantages does it have compared to existing solutions?

  • From a research perspective, this study is the first to explore the collaborative role of GenAI in online debates from a multi-user interaction angle.
  • It provides insights into designing more effective GenAI interaction systems for debate scenarios.

• What were the experimental or evaluation results?

  • Data analysis revealed that certain behaviors in collaboration with ChatGPT (e.g., hasty adoption of statistical data) might unintentionally undermine the critical depth of debates.
  • In freer debate modes, human users significantly reduced their reliance on generative AI to better interact with others.

• Limitations and Future Directions

  1. Limitations:

     • The study recruited participants with high educational backgrounds, most of whom were non-native English speakers (from Asia), which may limit the generalizability of the results.
     • The study restricted users to text-only communication, excluding other expressive forms like video and images, which may be indispensable in real-world online debates.

  2. Future Directions:

     • Expand the study to larger-scale, more diverse participant communities.
     • Explore how to design interfaces that more naturally integrate GenAI functionalities and optimize human-AI collaboration efficiency.
     • Incorporate more diverse online forums that allow multimodal content, making the scenarios closer to real-world applications.

Conclusion

This study delves into the role of generative AI in online debates, revealing how users flexibly utilize ChatGPT to assist debates while emphasizing their strong desire to maintain control over the debate and the dynamic changes in AI usage frequency in multi-user interaction contexts. These findings provide valuable insights for designing future AI debate assistants that better balance efficiency and human-centric needs.