How to Support Users in Understanding Intelligent Systems? Structuring the Discussion

Title of the Paper

How to Support Users in Understanding Intelligent Systems? A Structured Discussion

Paper Information

• Field of Study: Human-Computer Interaction (HCI), User Experience and Explainability in Intelligent Systems
• Keywords: Intelligent Systems, Explainability, Transparency, Auditability, Clarity, Accountability, Interactive Machine Learning, End-User Debugging

Research Background and Questions

• Research Background

  • The "black-box" nature of intelligent systems (i.e., unpredictable outcomes and difficulty in error correction) violates traditional user interface design principles, making them challenging to design, understand, and use.
  • Increasingly, researchers, practitioners, and policymakers are calling for improved transparency, explainability, and auditability in intelligent systems to support user understanding.
  • Existing research on system attributes such as transparency and explainability often suffers from inconsistent terminology and conceptual definitions, hindering further progress in the field.

• Research Questions

  • What are the implicit assumptions researchers make about system attributes when users interact with intelligent systems?
  • How can these assumptions be structured and differentiated?

• Significance of the Research

  • Addressing the "black-box" problem in intelligent systems is critical for enhancing user trust and improving user experience through optimized interaction design.
  • Clarifying the ambiguity in terminology and concepts contributes to the rigor and coherence of research, supporting the development of more explainable and user-friendly intelligent systems.

Solutions

• Method Overview

  • This paper reviews user-related issues in HCI literature and proposes a conceptual framework by synthesizing user mindsets, forms of user involvement, and types of knowledge acquisition. The aim is to provide a structured discussion on how to support users in understanding intelligent systems.

• Core of the Framework

  1. User Mindsets: Exploring what aspects of the system users wish to understand.
     • Practicality: Users seek to understand and predict system behavior to achieve specific goals.
     • Interpretability: Users want to comprehend the relationship between system outputs and their own experiences.
     • Criticality: Users focus on the legal, ethical, and societal impacts of the system.
  2. Forms of User Involvement: Exploring the direction of information exchange.
     • Active Mode: Users provide input or feedback to the system (e.g., debugging and correcting errors).
     • Passive Mode: Users receive information from the system (e.g., explanations of how decisions are made).
  3. Knowledge Outcomes: Clarifying the types of knowledge users acquire from the system.
     • Output Knowledge: Understanding specific outputs (e.g., recommendation results).
     • Process Knowledge: Overall understanding of system operations (e.g., neural networks).
     • Interaction Knowledge: How users operate the system (e.g., providing feedback).
     • Metacognitive Knowledge: Knowledge beyond the direct interaction context (e.g., system development background).

• Innovations

  • The framework links user questions to the diverse attributes of intelligent systems, providing a unified perspective to organize and compare existing research.
  • It proposes a practical and rigorous approach to uncover and address conceptual ambiguities in the field.

• Implementation

  1. Literature Sampling: Reviewing 222 papers related to key system attributes such as transparency and explainability.
  2. Coding and Interpretation: Using open coding, axial coding, and selective coding centered on user questions to develop the framework.
  3. Framework Validation: Demonstrating the framework's operability and explanatory power through practical application examples.

Research Outcomes

• Framework Application

  • The framework is suitable for reviewing and categorizing existing work and can guide the design of better user interfaces and system interaction models in the future.
  • For example, by mapping user mindsets to specific applications (e.g., interaction design for recommendation systems), targeted optimizations can be achieved.

• Advantages Over Existing Solutions

  • The framework clearly reveals implicit conceptual discrepancies and connections in existing literature, facilitating the standardization of terminology.
  • It supports researchers in posing the right questions from the user's perspective to design intelligent interfaces that better meet practical needs.

• Experimental Validation

  • Provides a comprehensive classification of knowledge acquisition outcomes (output, process, interaction, and metacognitive knowledge).
  • Offers specific design recommendations (e.g., multi-modal explanation interfaces) to support user needs in different usage scenarios.

• Limitations and Future Directions

  • The framework has certain limitations, such as a dataset biased toward ACM Digital Library citations and an inability to cover all possible intelligent system attributes.
  • Future work should further validate the framework's value in real-world product design and commercial applications.
  • Cross-disciplinary collaboration (e.g., communication studies, policy research, ethics) is recommended to promote a more comprehensive understanding of societal impacts.