Are Explanations Helpful? A Comparative Study of the Effects of Explanations in AI-Assisted Decision-Making

Title of the Paper

Are Explanations Helpful? A Comparative Study of the Effects of Explanations in AI-Assisted Decision-Making

Paper Information

• Subject Area: Effectiveness of Explainable Artificial Intelligence (XAI) in human-AI collaborative decision-making
• Keywords: Explainable AI, machine learning explanations, trust calibration, uncertainty awareness, human-computer interaction

Research Background and Problem

• Identified Problems or Challenges:

  1. The black-box nature of AI models limits users' trust in and adoption of model predictions.
  2. Although many XAI methods have been developed, there is a lack of systematic studies evaluating their practical effectiveness in different contexts.

• Significance: AI-driven decision support is widely applied in critical societal domains (e.g., justice and environmental management), and the effectiveness of model explanations directly impacts the outcomes of human-AI team decisions.

• Research Motivation and Related Work:

  • Existing studies often define the "quality" of explanation methods based on developers' intuition, lacking evaluations from the perspective of user behavior.
  • Social science literature suggests that humans prefer comparative explanations, but there is insufficient experimental validation of these methods' utility.

Solution

• Methods or Solutions:

  1. Propose three ideal properties of AI explanations (termed "explanation needs"):
  • Understanding: Does the explanation improve users' understanding of the model?
  • Uncertainty Awareness: Can it help users recognize the uncertainty in model predictions?
  • Trust Calibration: Does it support users in appropriately trusting the model?
  1. Experimentally evaluate the performance of four mainstream model-agnostic explanation methods in two decision-making tasks through randomized controlled trials:
  • Feature Importance
  • Feature Contribution
  • Nearest Neighbors
  • Counterfactual Explanations

• Innovations:

  • Introduced the variable of users' perceived domain knowledge differences (low vs. high) into the experimental design to assess the variability in explanation effectiveness across tasks.
  • Proposed a clear evaluation framework for explanation needs, providing a reference standard for future explanation method design and validation.

• Implementation Steps:

  1. Select two real-world tasks with differing domain knowledge requirements: recidivism risk prediction and forest cover prediction.
  2. Conduct randomized group experiments on the Amazon MTurk platform to compare whether different explanation methods meet the "explanation needs."
  3. Analyze experimental results to reveal the relationship between explanation effectiveness and task characteristics.

Research Findings

• Specific Findings:

  • The fulfillment of "explanation needs" varied significantly across the two tasks:

    • Low domain knowledge task (forest cover prediction): Most explanation methods failed to meet the "explanation needs."
    • High domain knowledge task (recidivism prediction):
      • Feature Contribution explanations better satisfied the needs for "Understanding" and "Trust Calibration."
      • Counterfactual explanations improved model understanding but did not support trust calibration.

  • Provided quantitative analysis tools and qualitative insights to guide the development of future explanation methods.

• Strengths:

  • The authors conducted a systematic, direct comparison of four classical explanation methods, validating their practical benefits from a user behavior perspective.
  • Highlighted the critical influence of domain knowledge on the effectiveness of explanation methods, offering valuable insights for explanation design.

• Experimental or Evaluation Results:

  • Feature Contribution explanations enhanced users' understanding and reliance on high-confidence predictions, while other explanations (e.g., Counterfactuals, Nearest Neighbors) showed limited ability to support trust calibration.
  • The higher the users' domain knowledge, the more significant the positive impact of explanations.

• Limitations and Future Directions:

  • The current experiments focused on a simple logistic regression model, which may not fully reveal the performance of explanation needs for more complex models.
  • The visualization design of the explanations in the experiments was limited; future studies should explore more dynamic and progressive explanation presentation methods.
  • Further research is recommended to investigate how other task characteristics (e.g., model accuracy) moderate the effects of explanations.

Conclusion

This study systematically compared the effectiveness of various mainstream XAI methods in different task contexts, providing new insights for XAI method design. The findings highlight the critical impact of domain expertise on explanation effectiveness, offering clear directions for designing explanations that minimize cognitive load and present information intuitively.