Contextualization and exploration of local feature importance as explanations to improve understanding and satisfaction of non-expert users

Title of the Paper

Contextualization and Exploration of Local Feature Importance Explanations to Improve Understanding and Satisfaction of Non-Expert Users

Paper Information

• Subject Area: Artificial Intelligence, Explainability of Machine Learning Models, and Human-Computer Interaction
• Keywords: Explainable Artificial Intelligence (XAI), Human-Centered Approach, User Study, Interface Design, Local Feature Importance

Research Background and Problem Statement

• Identified Problems or Challenges:

  • As machine learning (ML) models are increasingly used in decision support, their lack of interpretability limits their adoption, especially for non-expert users.
  • Local feature importance explanations (e.g., SHAP methods) are tools to help non-expert users understand specific model outputs, but studies show that such explanations can lead to user misunderstandings or incomplete mental models.
  • Non-expert users' comprehension of such explanations is influenced by presentation formats (e.g., charts vs. text).

• Significance of the Research:

  • Providing clear and transparent model explanation mechanisms can enhance users' trust and satisfaction with machine learning models while reducing risks from misinterpretations.
  • Explanation designs optimized for non-expert users can promote the broader adoption of AI technologies.

• Motivation and Related Work:

  • Current design methods aimed at explainability mostly target ML or domain experts rather than ordinary users lacking technical backgrounds.
  • Research in social sciences and XAI suggests that contextualization and exploratory interfaces have potential in improving user understanding.
  • Existing literature has explored enhancing visualization of explanations through prior knowledge and example comparisons but lacks extensive validation for non-expert users.

Proposed Solution

• Proposed Method or Solution:

  • Introduced general XAI design principles focusing on "contextualization" and "exploratory affordances" of local feature importance explanations from the perspective of non-expert users.
  • Contextualization design includes providing ML transparency, domain transparency, and external transparency.
  • Exploratory design incorporates interactive displays and example-based explanations.

• Innovative Aspects:

  • Tailored to the needs of non-expert users, combining contextual information and interactive tools to enhance explanation usability.
  • Highlighted the importance of clearly conveying external factors such as gender and vehicle information in the user interface to avoid misunderstandings.

• Implementation Steps and Techniques:

  • Developed an interactive prototype in the context of car insurance pricing to display model predictions and explanations, using the SHAP method to generate local feature importance explanations.
  • Designed three specific interface principles: card-based displays, transparent descriptions of model/domain/external information, and the inclusion of interactive buttons and static example charts.

Research Outcomes

• Specific Outcomes:

  • Designed and implemented an interactive prototype based on the proposed principles for explaining customized car insurance pricing.
  • Validated the impact of these improvements on non-expert users' understanding and satisfaction through user experiments.
  • "Contextualization" significantly improved users' subjective satisfaction and showed a near-significant improvement in objective understanding (p=0.06).
  • "Exploratory affordances" significantly enhanced user satisfaction (p=0.03).

• Comparative Advantages Over Existing Solutions:

  • The proposed design is more user-friendly for non-expert users, not just for experts.
  • Simultaneously optimized users' understanding (objective dimension) and satisfaction (subjective dimension), reducing users' operational time costs.

• Experimental or Evaluation Results:

  • A total of 80 users participated in the experiment, testing understanding and satisfaction under four conditions (including a baseline condition without contextualization or exploratory features).
  • Interfaces with contextualization increased user satisfaction scores from 3.65 to 4.57; combining exploratory features further raised scores to 4.79.

• Limitations and Future Directions:

  • Limitations:

    • The current experiment focuses on a single application domain (car insurance), limiting its generalizability to other scenarios.
    • The sample size is insufficient for in-depth comparative analysis of multiple contextualization principles.

  • Future Directions:

    • Extend the study to other industry domains (e.g., healthcare, recruitment) to validate the generalizability of the findings.
    • Explore the potential relationship between user understanding and satisfaction.
    • Increase the number of experimental participants to investigate the dependency between user characteristics and explanation effectiveness.