Debiased-CAM to mitigate image perturbations with faithful visual explanations of machine learning

Title of the Paper

Debiased-CAM to Mitigate Image Perturbations with Faithful Visual Explanations of Machine Learning

Paper Information

• Domain: Explainability and Robustness in Machine Learning
• Keywords: Explainable AI, Misleading Explanations, Class Activation Map, Robust Machine Learning, Image Perturbations, User Study

Research Background and Problem

• Identified Problems or Challenges:
  • Class Activation Maps (CAMs), as a commonly used visualization method, help users understand the behavior of deep learning models. However, when input data is affected by systematic biases (e.g., image blur, color distortion, or lighting changes), CAMs deviate from the original true targets and become misleading.
  • Conventional model fine-tuning (e.g., training with biased input data) improves prediction performance but often fails to produce explanations consistent with the original scenarios.
  • Many current Explainable AI (XAI) techniques lack support for clear and faithful explanations under "biased" conditions.
• Significance:
  • To enhance user trust in AI systems, model predictions need to be accurate, and the generated explanations must align with user cognition. Otherwise, trust may be undermined, and users may be misled.
• Research Motivation and Related Work:
  • Given the limited effectiveness of existing models and explanation methods on biased data (even techniques like explanation transfer fail to alleviate biases), the authors designed a novel training strategy to improve the trustworthiness of explanations.
  • This work incorporates user studies to evaluate the impact of bias on explanation fidelity and usefulness, and to validate the effectiveness of debiased explanations.

Solution

• Proposed Method or Solution:
  • A method called "Debiased-CAM" is proposed to generate model explanations on biased data that are closer to the true targets.
  • A multi-input, multi-task convolutional neural network (DebiasedCNN) is designed, leveraging auxiliary tasks (e.g., explanation prediction and bias level prediction) to achieve self-supervised learning.
• Innovations:
  • Treating explanation as a secondary prediction task, the model uses self-supervised training to "de-bias" biased images, ensuring that the generated explanations are closer to the results of unbiased images.
  • The model not only learns to produce more robust predictive performance but also ensures explanation fidelity. Additionally, this framework supports generalization across different types and levels of bias.
  • Multiple model variants were designed to systematically evaluate the contributions of single/multiple biases and single/multi-task setups.
• Implementation Steps and Key Techniques:
  1. Generate Class Activation Maps (Unbiased-CAM) from unbiased images and use them as a reference for CAM fidelity during training with biased images.
  2. Optimize using self-supervised learning to minimize the explanation deviation between Unbiased-CAM and Debiased-CAM, while maintaining high task performance on biased data.
  3. Enhance the model's bias awareness and robustness through an additional bias level prediction task.

Research Outcomes

• Specific Results:
  • DebiasedCNN significantly improved prediction accuracy on biased data while generating more trustworthy explanations.
  • Adding the bias level prediction task enabled the model to maintain high consistency and performance across different bias levels.
  • User studies revealed that compared to standard CAM, Debiased-CAM was perceived as more truthful and helpful, reducing user trust misdirection caused by erroneous explanations.
• Advantages:
  • Compared to existing methods, Debiased-CAM not only improves task performance on biased data but also addresses user understanding and trust issues through explanation bias adjustment.
  • The method is modular and potentially extendable to other XAI domains (e.g., temporal data, audio data).
• Experimental or Evaluation Results:
  • Simulated studies validated the impact of bias types (e.g., blur, color temperature changes, lighting changes) on explanation performance and confirmed the effectiveness of Debiased-CAM across various tasks (e.g., classification, image description).
  • Under the same bias levels, Debiased-CNN achieved several-fold improvements in explanation and prediction performance compared to baseline models.
  • User studies demonstrated that debiased explanations significantly enhanced users' ability to verify model accuracy under visually blurred task conditions.
• Limitations and Future Directions:
  • The definition and scope of data bias may limit model performance; for instance, certain complex adversarial noise may make it difficult for Debiased-CAM to learn bias corrections.
  • The application of Debiased-CAM to explanations involving social biases was not explored.
  • Future work could extend to more complex explanation methods, multi-dimensional data, or integrate generative networks to further enhance explanation fidelity.

Summary and Design Implications

• This study highlights the impact of data bias on explanation reliability and proposes Debiased-CAM as an important method to address this issue.
• The developed debiasing training framework based on self-supervised learning demonstrates that both model explanations and performance on biased data can be simultaneously improved, opening up possibilities for AI tasks in ambiguous scenarios.
• For practical applications, the authors recommend that researchers consider the potential impact of bias on user trust when developing explanation algorithms and incorporate user testing to enhance the applicability and transparency of explanations.