EXMOS: Explanatory Model Steering through Multifaceted Explanations and Data Configurations

Title of the Paper

EXMOS: Explanatory Model Steering Through Multifaceted Explanations and Data Configurations

Paper Information

• Domain: Applications of Explainable Artificial Intelligence (XAI) and Interactive Machine Learning (IML) in high-risk domains such as healthcare
• Keywords: Explainable AI, XAI, Interactive Machine Learning, IML, Explanatory Interactive Learning, Interpretable AI, Responsible AI, Model Steering

Research Background and Problem

• Identified Issues or Challenges:
  Whether current explanation methods in machine learning models can effectively assist domain experts (e.g., medical professionals) in data adjustments and model optimization remains underexplored. In particular, the impact of global model-oriented explanations and data-oriented explanations on guiding users in data configuration is unclear.
• Significance:
  In high-risk domains such as healthcare, the transparency and explainability of AI systems are crucial for enabling domain experts to understand and trust these systems, especially when they need to adjust training data and optimize models through feedback mechanisms.
• Motivation and Related Work:
  1. Existing research shows that data-oriented explanations can reveal biases, inconsistencies, and quality issues in training data, but these studies mainly focus on explanations for single prediction instances (local explanations).
  2. Model-oriented explanations (e.g., feature importance analysis) are widely used but may lack actionable insights for domain experts.
  3. Combining domain knowledge with model optimization is essential in high-risk domains, but user studies in this area are still scarce.

Proposed Solution

• Proposed Approach:
  The authors propose a system called EXMOS, which integrates data-oriented and model-oriented explanations to assist domain experts in optimizing machine learning model performance through manual and automated data configuration mechanisms.
• Innovations:
  1. Investigated the impact of different types of global explanations (data-oriented, model-oriented, and hybrid explanations).
  2. Introduced manual and automated data configuration mechanisms, enabling domain experts to flexibly adjust training data and immediately observe model changes.
  3. Provided design guidelines to enhance the effectiveness and efficiency of explanation-driven interactive machine learning systems.
• Implementation Steps and Key Techniques:
  1. Explanation Dashboard Design:
     • Data-oriented explanations: Key data insights, variable distribution charts (data quality indicators), overall training data quality evaluation.
     • Model-oriented explanations: Feature importance, key decision rules.
     • Hybrid explanations: Combined visual components of data-oriented and model-oriented explanations.
  2. Data Configuration Mechanisms:
     • Manual configuration: Users filter data variables or adjust variable ranges to select relevant predictive variables.
     • Automated configuration: Detects and corrects issues in training data (e.g., outliers, class imbalance), effectively reducing data quality problems.

Research Outcomes

• Specific Findings:

  1. Quantitative Results:
     • Users in the hybrid explanation dashboard (HYB) group significantly improved model performance (notably increased model accuracy), although task load was higher.
     • Data-oriented explanations outperformed model-oriented explanations in understanding data layout and guiding data configuration, significantly improving system transparency.
     • Users found manual configuration mechanisms more efficient than automated ones, as manual configuration offered greater flexibility and control.
  2. Qualitative Results:
     • Data-oriented explanations enhanced users' understanding of the system, effectively facilitated the identification of data issues, and fostered trust in the system.
     • Manual configuration enabled users to explore data, validate domain knowledge, and provided higher actionability.
     • Automated configuration was perceived as time-saving but required explanation of the automatic correction mechanisms to improve transparency.
  3. The study highlighted the insufficiency of global model-oriented explanations in guiding users for data configuration and emphasized the importance of combining data-oriented and model-oriented explanations.

• Advantages:

  1. The hybrid explanation approach allowed users to gain a more comprehensive understanding of the model and data, thereby optimizing the model more effectively.
  2. Incorporating domain experts' feedback mechanisms enhanced the reliability and personalization of model optimization.

• Limitations and Future Directions:

  1. Limitations:
     • Participant samples were limited to a single institution and younger subjects, with limited feedback from older populations.
     • The specific impacts of manual and automated data configuration mechanisms were not separately studied.
  2. Future Directions:
     • Conduct randomized controlled studies to further explore the independent effects of data configuration mechanisms on user trust and understanding.
     • Combine local and global explanations for optimization studies in medical scenarios.
     • Investigate the application of conversational explanations and interactive mechanisms.

Conclusion

This study provides a new perspective on enhancing model performance in healthcare through transparent explanations and robust user feedback mechanisms. The authors proposed a set of design guidelines and demonstrated the effectiveness of the EXMOS system, offering valuable references for the design and user research of AI systems in high-risk domains.