Explainable Moderation in Multiplayer Games: Player Responses to Explanations of an Automated Temporary Ban

Paper Title

Explainable Moderation in Multiplayer Games: Player Responses to Explanations of an Automated Temporary Ban

Publication Info

• Topic area: Explainable AI in automated moderation for multiplayer games.
• Keywords: Explainable AI, automated moderation, multiplayer games, toxicity, fairness, emotional response, player agency, community management, transparency, behavior change.

Background and Problem

• Problem / challenge: Current moderation systems in multiplayer games are opaque, leaving punished players confused and without guidance for behavior change. Explanations provided are often simplistic and fail to address players' informational needs.
• Significance: Addressing opaqueness in moderation systems can improve perceptions of fairness, emotional responses, and potentially foster behavior change, leading to healthier online communities.
• Motivation and related work: Prior research highlights the challenges of over-reliance on automation, punitive governance, and lack of transparency in moderation systems. While explainable AI has shown promise in improving transparency and fairness, no prior work has systematically evaluated different types of explanations in the context of automated game moderation.

Solution

• Proposed approach: Evaluation of six explanation types for automated moderation decisions in multiplayer games, including industry-standard and explainable AI methods, to assess their impact on perceived fairness, emotional response, and perceived utility.
• Novelty:
  1. Systematic comparison of six explanation types, including novel explainable AI methods, in the context of multiplayer game moderation.
  2. Identification of justification (providing evidence) as a critical factor for improving fairness and emotional responses.
  3. Reflexive thematic analysis revealing four themes in players' perceptions of moderation and explainability.
  4. Design implications for integrating explainability into moderation systems.
• Procedure and key techniques:
  1. Conducted a mixed-methods online experiment with 43 participants.
  2. Evaluated six explanation types: unjustified, basic, feature-based, case-based, counterfactual-based, and exploration-based.
  3. Measured perceived fairness, emotional response, and perceived utility using Likert scales, the Self-Assessment Manikin test, and open-ended questions.
  4. Performed quantitative analysis (Friedman tests and Wilcoxon signed-rank tests) and qualitative reflexive thematic analysis.

Results

• Concrete findings:
  1. Justified explanations (those providing evidence) were perceived as significantly fairer and elicited more positive emotional responses than unjustified explanations.
  2. Explainable AI methods showed minor benefits over the basic explanation in terms of understanding and feelings of control but did not significantly improve overall fairness or emotional response.
  3. Exploration-based explanations were the most preferred, followed by feature-based explanations.
• Advantage over baselines:
  • Justified explanations outperformed unjustified ones in fairness and emotional response.
  • Feature-based explanations improved understanding compared to the basic explanation.
  • Exploration-based explanations enhanced feelings of control.
• Experiments / evaluation:
  • Participants evaluated six explanation types within a controlled scenario involving a temporary ban for offensive language in a fictional multiplayer game.
  • Dependent variables included perceived fairness, emotional response, and perceived utility.
  • Quantitative and qualitative data were collected and analyzed.
• Limitations and future work:
  • Limited generalizability due to underrepresentation of women and gender-diverse participants.
  • Hypothetical experimental context may not fully capture real-world player behavior.
  • Future work should explore preferences in specific games, co-design explanations with players, and examine links between explainability and contestability.

Summary

This study evaluates the role of explainability in automated moderation systems for multiplayer games by comparing six explanation types. Justified explanations significantly improved perceived fairness and emotional responses, with exploration-based and feature-based explanations being the most preferred. However, explainable AI methods offered only minor advantages over the basic explanation. The findings highlight the importance of justification and suggest design implications for integrating explainability into moderation systems, including aligning explanations with community values, calibrating evidence and communication, promoting player agency, and diversifying community management. These insights aim to foster more inclusive and transparent player communities.