Improving understandability of feature contributions in model-agnostic explainable AI tools

Title of the Paper

Improving the Interpretability of Feature Contributions in Model-Agnostic Explainable AI Tools

Bibliographic Information

• Subject Area: Explainable Artificial Intelligence (XAI), Human-Computer Interaction, Information Visualization
• Keywords: Explainable Machine Learning, Explanation, Argumentation, Natural Language, Model-Agnostic Tools, Feature Contributions, Interpretability, Positive Framing, Semantic Labeling

Research Background and Problem Statement

• Problem Description:

  1. Many artificial intelligence (AI) systems are characterized as "black-box" models, making it difficult for users to understand how or why specific decisions are made.
  2. Feature contribution explanations in XAI tools (e.g., SHAP and LIME) often use positive and negative values, which can impose cognitive burdens on lay users and reduce the interpretability of model outputs.
  3. In existing tools, the framing of feature contributions (i.e., the meaning of positive and negative values) is typically based on model-predicted classes rather than user-perceived "positive" or "negative" outcomes, potentially causing intuitive confusion.
  4. Numerical feature contributions require users to infer the directional impact of these contributions on decisions, further increasing cognitive load.

• Significance of the Research: As AI systems are increasingly applied in domains like finance and healthcare that directly impact public interest, improving the interpretability of algorithmic decisions and user comprehension is critical. Furthermore, European laws have explicitly recognized consumers' right to explanations, underscoring the practical importance of this issue.

• Motivation and Related Work:

  1. In tools like SHAP and LIME, the presentation of contribution values can lead to misunderstandings, such as positive bar charts representing undesirable outcomes for users (e.g., loan rejection).
  2. Cognitive psychology studies suggest that people process positive information more easily and are better able to understand information with clear semantic labels.
  3. The authors propose two potential improvements—"Positive Framing" and "Semantic Labeling"—to address these issues.

Solution

• Methods or Solutions:

  1. Positive Framing: Present feature contributions in a user-centric positive outcome framework rather than a positive-negative framework based on model decision classes. For example, restructure "contributions" so that positive values always represent outcomes desired by the user.
  2. Semantic Labeling: Use explicit language to describe the specific impact of features on decisions, reducing the cognitive reasoning steps required by users. For instance, replace "+5%" with "+5% eligibility improvement" to make the feature's meaning immediately clear.

• Innovations:

  1. Breaks away from the default framing conventions of existing tools by defining positive meanings from the user's perspective for the first time.
  2. Proposes a textual semantic labeling method to enhance intuitive understanding and reduce user learning costs.
  3. Leverages interdisciplinary theories (e.g., cognitive psychology) to optimize XAI tool design.

• Implementation Steps:

  1. Generate visual explanations using improved tools based on positive framing and semantic labeling.
  2. Evaluate the impact of these improvements on interpretability through user studies.
  3. Conduct step-by-step experiments comparing the baseline (e.g., SHAP and LIME default frameworks) with the proposed improvements.

Research Findings

• Summary of Results:

  1. Effectiveness of Positive Framing:
     • Positive framing significantly improved interpretability regardless of whether the prediction was positive or negative. This indicates that redefining the framework can reduce users' cognitive confusion.
  2. Effectiveness of Semantic Labeling:
     • Adding semantic labeling improved interpretability, especially when positive language was used (e.g., "eligibility" was easier to understand than "ineligibility").
     • When semantic labeling was present, the impact of framing was significantly reduced (i.e., the role of positive-negative framing became less pronounced).
  3. Comparison with Existing Solutions:
     • The improved methods significantly outperformed existing solutions (e.g., SHAP and LIME default mechanisms) in enhancing interpretability and usability for both technical and non-technical users.
  4. Tool Implementation:
     • The authors integrated their findings into the ArgueView toolkit, which supports the automatic generation of more interpretable explanation visualizations.

• Experiments and Evaluation Results:

  1. A user study involving 133 participants found that positive framing improved interpretability scores, particularly in the absence of semantic labeling (p < 0.001).
  2. When positive semantic labeling was added, users demonstrated high levels of comprehension even when the framing was negative.
  3. The statistical significance of the academic survey was strong (α < 0.05), demonstrating robust cognitive benefits.

• Limitations and Future Directions:

  1. Limitations:
     • The study revealed a low degree of personalization in task contexts (e.g., hypothetical loan scenarios and music recommendations based only on user preferences), which may limit the generalizability of results to real-world applications.
     • The experimental design did not cover more complex scenarios, such as multi-feature interactions and highly technical XAI applications.
  2. Future Directions:
     • Explore the effects of positive framing and semantic labeling in other domains (e.g., medical applications).
     • Further quantify the sources of cognitive load and develop automated algorithms to determine which decision categories are perceived as positive by users.
     • Optimize general-purpose explanation tools by integrating multimodal data (e.g., text, speech) and complex AI models.