Beyond Expertise and Roles: A Framework to Characterize the Stakeholders of Interpretable Machine Learning and their Needs

Title of the Paper

Beyond Expertise and Roles: A Framework for Describing Stakeholders and Their Needs in Interpretable Machine Learning

Paper Information

• Subject Area: Interpretable Machine Learning
• Keywords: Interpretability, Explainability, Machine Learning, Knowledge, Needs, Goals, Framework

Research Background and Problem

• Identified Issues or Challenges:

  • Current machine learning systems are often described as "black boxes," making it difficult for users to understand their logic and build trust.
  • Existing research primarily describes stakeholders based on expertise and roles, failing to fully reflect the diversity and specificity of participants' needs.
  • Interpretability methods often lack clarity about their target users, making it challenging to address diverse needs in design.

• Significance:

  • As machine learning models are increasingly applied across various domains, they significantly impact societal and individual decision-making. Ensuring the interpretability of their outputs is crucial for building trust, achieving fair decisions, and mitigating adverse effects.

• Research Motivation and Related Work:

  • Previous studies have attempted to define user needs by categorizing them as "experts" or "non-experts" or by functional roles, but these approaches are limited in their descriptive and generative capabilities.
  • A more nuanced classification framework is needed to better understand stakeholders' knowledge backgrounds and needs, thereby improving user research and method design.

Proposed Solution

• Proposed Method or Solution:

  • The researchers developed a framework to describe stakeholders and their needs in interpretable machine learning, comprising two main components:
    1. Decomposing stakeholders' knowledge into three types: formal knowledge (theoretical knowledge), tool knowledge (practical skills), and personal knowledge (experience and values).
    2. Defining stakeholders' needs as a three-tier classification: long-term goals (e.g., understanding the model, building trust), mid-term goals (e.g., debugging the model or ensuring compliance), and specific tasks (e.g., assessing prediction reliability, detecting errors).

• Innovative Aspects:

  • The framework goes beyond categorizing by roles or expertise by introducing types of knowledge and their application contexts (machine learning, data domains, broader societal environments).
  • It eliminates oversimplified descriptions of stakeholder needs, revealing that needs can span across different roles and expertise levels.

• Implementation Steps and Key Techniques:

  • Literature review and open coding: Analyzing literature from various fields (machine learning, educational theory, social sciences) to extract descriptions of stakeholder needs.
  • Developing the framework using iterative analysis and architecture optimization methods.
  • Validating the framework's descriptive capabilities with 58 related papers and exploring its evaluative and generative potential.

Research Outcomes

• Specific Outcomes:

  • The proposed framework effectively encompasses the primary stakeholders and needs in the field of interpretable machine learning, highlighting gaps in existing research, such as the limited attention to non-technical user needs.
  • It demonstrates that the framework can aid in designing more precise user testing and clarifying the relationships between different tasks.
  • The framework generates new combinations of user roles and needs, offering guidance on improving the design of interpretability interfaces.

• Advantages:

  • Compared to existing classification methods based on roles or expertise, this framework is more descriptive and generative, uncovering new stakeholder needs and research directions.
  • It provides stakeholders with a more detailed vocabulary, supporting the design of model explanation interfaces that better align with application contexts.

• Experimental or Evaluation Results:

  • Coding 58 papers revealed that the framework comprehensively describes diverse knowledge backgrounds and needs.
  • New combinations of user roles and needs were generated, such as how atypical technical users with personal knowledge influence model debugging.

• Limitations and Future Directions:

  • The framework does not yet cover all needs, such as the less prominent "persuasion and adoption" needs.
  • Future work could further refine the classification of personal knowledge and social contexts to reflect a broader range of needs.
  • Verbalizing user needs and stakeholder knowledge may be insufficient; more systematic methods are needed to capture hidden stakeholder needs.

This framework not only provides a more detailed and practical tool for interpretability research but also helps design machine learning explanation methods that better meet real-world needs. It opens new avenues for reflecting on the design process from social and user perspectives.