Comparables XAI: Faithful Example-based AI Explanations with Counterfactual Trace Adjustments

Paper Title

Comparables XAI: Faithful Example-based AI Explanations with Counterfactual Trace Adjustments

Publication Info

• Topic area: Explainable AI (XAI) methods for improving user understanding of AI decisions.
• Keywords: Explainable AI, example-based explanations, counterfactuals, trace adjustments, faithfulness, interpretability, valuation tasks, user studies, modeling study, decision support.

Background and Problem

• Problem / challenge: Existing XAI methods are often too technical for non-expert users, and example-based explanations lack faithfulness when comparables differ significantly from the subject. Linear adjustments are simplistic and fail to capture non-linear trends in complex domains.
• Significance: Faithful and interpretable explanations are critical for fostering trust and understanding in AI systems, particularly in domains requiring analytic comparison across multiple attributes, such as real estate valuation and salary estimation.
• Motivation and related work: Example-based explanations are intuitive but imprecise for complex domains. Counterfactual explanations are analytical but often focus on actionable recourse rather than faithful estimation. Prior methods like linear regression and linear adjustments fail to balance faithfulness and interpretability in non-linear contexts.

Solution

• Proposed approach: Comparables XAI with Trace Adjustments—a method that incrementally adjusts attributes of comparables along the AI model's decision surface to generate faithful counterfactual traces.
• Novelty:
  1. Introduction of Comparables XAI as an explanation paradigm inspired by real estate valuation practices.
  2. Development of Comparables with Trace Adjustments to generate incremental counterfactuals for improved faithfulness and interpretability.
  3. Optimization of counterfactual traces using desiderata such as sparsity, disjointness, monotonicity, and evenness.
  4. Validation through modeling and user studies across multiple domains.
• Procedure and key techniques:
  1. Selection of comparables based on similarity to the subject.
  2. Incremental adjustment of attributes along the AI model's decision surface using a piecewise linear function.
  3. Optimization of trace adjustments with regularization terms for desiderata.
  4. Evaluation of the approach through modeling studies and user experiments.

Results

• Concrete findings:
  • Comparables with Trace Adjustments achieved the lowest prediction error, highest faithfulness, and narrowest uncertainty bounds across all tested domains.
  • User studies showed improved decision accuracy and confidence with Trace Adjustments compared to baselines.
• Advantage over baselines:
  • Outperformed Comparables Only, Comparables with Linear Regression, and Comparables with Linear Adjustments in faithfulness and precision.
  • Balanced accuracy and interpretability better than traditional methods.
• Experiments / evaluation:
  • Modeling study across five domains (house price, salary, energy consumption, drug sensitivity, crop yield) with metrics such as prediction error, unfaithfulness, and uncertainty bounds.
  • Formative user study with 25 participants to understand usability and perception of explanations.
  • Summative user study with 330 participants to evaluate decision accuracy, precision, and perceived helpfulness.
  • Expert user study with 5 real estate professionals to assess practical applicability.
• Limitations and future work:
  • Assumes preselected comparables; future work could explore methods for automatic selection.
  • Limited experimentation on confounding factors like prediction errors and counterfactual directionality.
  • No formal theoretical guarantees on faithfulness; future work could provide rigorous proofs.

Summary

Comparables XAI introduces a paradigm for generating faithful example-based explanations by tracing counterfactual adjustments along the AI decision surface. The method balances accuracy and interpretability through desiderata optimization. Modeling studies and user experiments demonstrate its effectiveness in improving decision accuracy, precision, and user confidence across multiple domains. While promising, future work is needed to address limitations in comparable selection, confounding factors, and theoretical guarantees. Comparables XAI has broad applicability in valuation and other decision-making contexts requiring analytic comparison.