(Mis)Communicating with our AI Systems

Research Background and Issues

• What problems or challenges did the authors identify?

  • Existing Explainable Artificial Intelligence (XAI) methods have not adequately addressed the issue of "miscommunication," where explanations fail to be accurately conveyed or interpreted between AI systems and humans.
  • XAI, as a communication process, fundamentally aims to connect AI inputs and outputs. However, the explanation process is influenced by human cognitive biases and social contexts, complicating the evaluation of its success.
  • Explanations provided to users with different backgrounds (e.g., developers, general users, domain experts) may fail to effectively meet their needs, resulting in comprehension barriers.

• Why is this issue important?

  • The ultimate goal of XAI is to enhance the transparency of AI decision-making processes and build user trust. In critical domains such as medical diagnosis and safety systems, miscommunication can lead to misunderstandings and even harm human interests.
  • Without establishing effective common ground, AI explanations may lead to misunderstandings, thereby affecting human trust in AI and the quality of decision-making.

• Research Motivation and Related Work

  • Drawing inspiration from research on human communication models in social sciences, such as feedback and the establishment of common ground, the authors aim to introduce these concepts into the XAI field to identify and reduce communication barriers.
  • The authors reference numerous related studies, such as the importance of human-machine dialogue (Guzman and Lewis, 2020) and the risks of manipulating user trust (Lakkaraju and Bastani, 2020). Additionally, prior research on social, contrastive, and selective XAI (Miller, 2019) provides a theoretical foundation.

Solution

• What methods or solutions did the authors propose?

  • The authors proposed viewing the XAI process as a cyclic communication model: incorporating feedback, common ground, and dynamic interaction between participants (human receivers and AI senders) to mitigate the risk of miscommunication.
  • They emphasized the need for XAI to establish "common ground," defined as shared knowledge, beliefs, and assumptions between participants.
  • Feedback mechanisms were introduced to the encoding (explanation generation) and decoding (explanation reception) phases of the XAI process to adjust and verify the accuracy of explanations.

• What is innovative about this solution?

  • The authors combined classical human communication models (e.g., Shannon & Weaver) with XAI and introduced human feedback mechanisms into AI explanations.
  • They defined "miscommunication" as a core issue and emphasized resolving it through feedback and common ground, a perspective that has not been sufficiently addressed in existing XAI research.

• What are the implementation steps and key technologies used?

  1. Mapping XAI inputs and outputs to key elements of the communication process: sender, receiver, information transmission, explanation decoding, etc.
  2. Defining specific explanation goals (e.g., trust, causality, informativeness) to guide explanation generation.
  3. Viewing current XAI methods (e.g., feature importance analysis, simplified models) as "information encoding" techniques and adding feedback mechanisms for validation and adjustment.
  4. Proposing an iterative explanation and validation feedback loop to identify potential misunderstandings or encoding errors.

Research Findings

• What specific findings were achieved?

  • The authors demonstrated the application of the cyclic communication model as a framework for XAI in two scenarios: music audio classification and medical diagnosis.
  • They proposed applying feedback to explanation decoding: for example, in music classification tasks, modifying input signals to verify explanation accuracy; in medical diagnosis, adjusting patient variables to test whether AI recommendations align with expectations.
  • They analyzed explanation goals and methods tailored to different user types (e.g., developers, domain experts).

• How does it compare to existing solutions?

  • The authors discussed how their approach makes AI explanations more dynamic and interactive, reducing user misunderstandings and enhancing trust in the system.
  • The proposed feedback mechanism significantly improves the reliability and acceptability of explanations, representing an advancement over existing XAI methods that rely on simple one-way explanations.

• What are the experimental or evaluation results?

  • The two case studies demonstrated how the framework helps identify misunderstandings in explanations and improves AI inference accuracy through feedback. For instance, iterative validation confirmed whether predictions aligned with actual classification model outputs, ensuring explanation reliability.

• Limitations and Future Directions

  • Limitations:
    • The model and methods are primarily based on theoretical analysis and have not yet undergone extensive external user testing in practical applications.
    • The focus is on single-instance explanations, with limited exploration of long-term and multi-instance interaction processes.
  • Future Work:
    • Designing and conducting user-centered experiments to validate the effectiveness of the cyclic communication model in real-world applications.
    • Expanding explanation methods to accommodate more complex user backgrounds and task requirements.
    • Considering the impact of more intricate cognitive biases (e.g., confirmation bias) on explanation selection to further optimize human-centric explanation mechanisms.