Wait, But Why?: Assessing Behavior Explanation Strategies for Real-Time Strategy Games

Title of the Paper

Wait, But Why?: Assessing Behavior Explanation Strategies for Real-Time Strategy Games

Paper Information

• Research Area: Human-Computer Interaction and Explainable Artificial Intelligence (XAI), Behavior Explanation in Esports
• Keywords: Explainable AI, esports, data-driven storytelling, real-time strategy games, multimedia interaction, cognitive load, behavior explanation, human-computer interaction, multi-screen applications, recall ability

Research Background and Problem

• Problem or Challenge:

  • People generally prefer "why" explanations and learn from them, yet esports commentators predominantly provide "what" descriptions.
  • Researchers aim to understand why "why" explanations are infrequent in real-time strategy esports commentary: Is it because audiences do not need "why" explanations, lack cognitive resources to process them in real-time, or because commentators struggle to produce "why" explanations in real-time?

• Significance:

  • With the widespread application of AI technologies and the prevalence of complex models, explainability is crucial not only for model trust but also for helping human users understand complex behaviors in practical applications.
  • Understanding how to optimize real-time explanation strategies can enhance the quality of esports commentary and audience comprehension while advancing research in explainable AI.

• Research Motivation and Related Work:

  • Lim and Dey's research shows that providing "why" explanations can significantly improve users' understanding and predictive abilities regarding complex systems.
  • Dodge et al. found that "what" questions dominate esports commentary, but aligning with Lim's research requires explaining why "why" questions are so rare.

Solution

• Proposed Research Method:

  • Evaluate the impact of three different methods on audience comprehension and cognitive load in an esports context: no explanation group, "what"-based explanation group, and "why"-based explanation group.
  • Use Dota 2 match videos and interactive maps to test the design and effectiveness of explanation strategies.

• Innovative Contributions:

  • Introduce a layered explanation strategy combining interactive textual prompts with multimedia elements.
  • Conduct the first experimental validation of the limitations of "why" explanations in real-time behavior explanation, proposing "what"-based explanations as an optimization strategy.

• Implementation Steps:

  1. Recruit 111 Dota 2 players for the experiment, randomly assigning them into three groups: watching regular videos, interactive maps without/with explanations.
  2. Provide explanations for 12 game events using interactive tools (varying in length and complexity for "what" and "why" explanations).
  3. Test cognitive load (using Paas and Van Merriënboer's nine-point scale) and audience recall ability for match events (seven multiple-choice questions assessing the impact of different explanations on comprehension).

Research Findings

• Key Results:

  • Audiences in the "what" explanation group performed significantly better in event recall tasks compared to the no explanation group.
  • The "why" explanation group experienced significantly higher cognitive load due to the complexity of explanations, resulting in weaker recall ability compared to the "what" explanation group.
  • Results indicate that providing detailed "why" explanations in real-time consumes excessive cognitive resources, while concise "what" explanations effectively enhance audience recall ability.

• Comparison with Existing Solutions:

  • Compared to relying solely on commentators, adding concise interactive "what" explanations offers a low-cost, cognitively friendly supplement for audiences.
  • Provides new experimental evidence supporting Dodge et al.'s theory on the limitations of "why" explanations in real-time contexts.

• Detailed Experimental Results:

  • Statistical analysis of recall tasks showed that the "what" explanation group had significantly higher accuracy than the no explanation group (p=0.0187).
  • Cognitive load test results indicated that the "why" explanation group had significantly higher load compared to the no explanation group (p=0.024).

• Limitations and Future Directions:

  • Limitations:
    • The experiment focused on the esports context of Dota 2, which may not directly generalize to other scenarios or applications.
    • The inefficiency of "why" explanations needs further validation to determine if it is constrained by real-time environments.
  • Future Directions:
    • Explore optimization of multimodal explanations (text, images, visualized data, etc.) in combined applications.
    • Develop models capable of automatically generating real-time explanations to reduce manual production costs.
    • Test the impact of explanation-based applications on audience learning outcomes beyond memory and comprehension tests.