Power Echoes: Investigating Moderation Biases in Online Power-Asymmetric Conflicts

Paper Title

Power Echoes: Investigating Moderation Biases in Online Power-Asymmetric Conflicts

Publication Info

• Topic area: Moderation biases in power-asymmetric online conflicts and the role of AI assistance.
• Keywords: Power asymmetry, moderation bias, human-AI collaboration, content moderation, large language models, consumer-merchant conflicts, heuristic thinking, social power, AI-generated suggestions, online platforms.

Background and Problem

• Problem / challenge: Existing research has extensively explored moderation biases in general conflict scenarios but has not systematically investigated biases in power-asymmetric conflicts. Additionally, the influence of AI-generated suggestions on these biases remains unclear.
• Significance: Understanding moderation biases in power-asymmetric conflicts is crucial for ensuring fairness and impartiality in online platforms, where such conflicts are prevalent (e.g., consumer-merchant disputes).
• Motivation and related work: Prior studies have shown that human moderators are influenced by social power cues, personal beliefs, and task pressures, leading to biased judgments. However, the specific biases in power-asymmetric conflicts and the role of AI in mitigating or amplifying these biases have not been systematically studied.

Solution

• Proposed approach: A mixed design experiment to investigate power-related biases in human and human-AI moderation of power-asymmetric conflicts, using real consumer-merchant conflict data.
• Novelty:
  1. First systematic study of moderation biases in power-asymmetric conflicts.
  2. Development of a taxonomy of power-related biases based on the "Bases of Social Power" theory.
  3. Examination of the impact of AI-generated suggestions on human moderation biases.
• Procedure and key techniques:
  • Collected 100 real conflict samples from the Dianping platform, categorized into 10 power manifestations (e.g., legitimate claim, punishment threat, compensation).
  • Conducted a mixed design experiment with 50 participants, divided into human-only and human-AI moderation groups.
  • Used a web-based program ("I Support") to present conflict samples and AI-generated suggestions (via Wizard-of-Oz design).
  • Analyzed judgment patterns, bias tendencies, and the influence of AI assistance through quantitative and qualitative methods.

Results

• Concrete findings:
  • Human moderation exhibited five significant power-related biases favoring the powerful party (e.g., legitimate claim, punishment threat, compensation, expert knowledge, length difference).
  • AI assistance alleviated most biases (four alleviated, one eliminated) but introduced an authority citation bias and amplified the legitimate claim bias.
  • AI-generated suggestions emphasizing reasons for unsupporting the opposite party shifted judgments toward the weaker party.
• Advantage over baselines:
  • AI assistance reduced reliance on heuristic cues (e.g., text length, professional terms) and provided alternative perspectives, improving fairness in some cases.
  • However, AI also reinforced biases when perceived as a source of verified authority.
• Experiments / evaluation:
  • Participants (n=50) completed 70 judgment tasks each, using initial and perturbed conflict samples.
  • Statistical analyses (e.g., paired t-tests) confirmed significant biases and the differential impact of AI assistance.
  • Semi-structured interviews provided qualitative insights into participants' perceptions and decision-making processes.
• Limitations and future work:
  • Limited to consumer-merchant conflicts on a Chinese platform, restricting cross-context and cross-cultural generalizability.
  • Wizard-of-Oz design may not fully replicate real-world AI outputs.
  • Future work should explore other power-asymmetric contexts, cultural variations, and real AI models.

Summary

This study systematically investigates moderation biases in online power-asymmetric conflicts, focusing on human and human-AI moderation. It identifies five significant biases favoring the powerful party and demonstrates that AI assistance can alleviate most biases but also introduces or amplifies some. Using real consumer-merchant conflict data and a mixed experimental design, the findings highlight the double-edged role of AI in shaping moderation judgments. These insights inform the design of fairer moderation systems and human-AI collaboration strategies, with implications for diverse online conflict scenarios. Future research should address cross-cultural generalizability and real-world AI applications.