The Amplifying Effect of Explainability in AI-assisted Decision-making in Groups

Research Background and Issues

• Identified Problems and Challenges:
  The paper highlights that AI-assisted decision systems are widely applied in critical domains such as healthcare, finance, and judicial systems. However, research on AI-assisted decision-making in group collaboration contexts remains limited. Furthermore, while Explainable Artificial Intelligence (XAI) is considered essential for enhancing AI system transparency, its effectiveness in specific scenarios often appears contradictory. For instance, studies on individual AI-assisted decision-making suggest that XAI improves decision performance, but its impact on group decision-making has yet to be thoroughly explored.

• Significance and Motivation:
  Group decision-making is extensively utilized in real-world scenarios, such as interdisciplinary medical team consultations or strategic planning by corporate boards. These tasks typically rely on collaboration among multiple individuals and have increasingly integrated AI-assisted tools. However, there is a lack of systematic research on the specific impact of AI on group decision-making and how explainability techniques may alter these effects. Research in this area is crucial for advancing human-AI collaboration and improving the efficiency of group tasks.

Solution

• Proposed Methods or Solutions:
  The paper investigates the impact of XAI on group AI-assisted decision-making and examines how group composition influences decision behaviors under conditions with or without AI explanations. The authors designed an experiment to test the performance of individuals and two-person groups in a mushroom edibility classification task under conditions with and without AI explanations.

• Innovations:
  This is the first quantitative study specifically focused on the impact of XAI in group decision-making. By exploring behavioral characteristics of over-reliance and under-reliance on AI, the study uncovers subtle differences in dependence on AI recommendations across varying group structures and XAI conditions. This research provides a new perspective for understanding the complex dynamics of group interactions in human-computer interaction.

• Implementation Steps and Techniques:

  1. Experimental Design: The mushroom edibility classification task was used, with two variables manipulated:

     • Group composition: individual decision-making versus two-person group decision-making.
     • Availability of AI recommendation explanations: XAI versus no XAI.

  2. Data Collection: Decision accuracy, decision time, subjective trust, and understanding of AI were recorded for each participant. Additionally, participants' reliance behaviors on AI recommendations were analyzed.

  3. Technical Support: XAI explanations were generated using the LIME algorithm, which clearly presented key information required for decision-making through feature importance.

Research Findings

• Specific Results:

  1. AI with explanations improved decision accuracy while reducing group over-reliance on incorrect AI recommendations.
  2. Groups without explanations were more prone to over-reliance, whereas the introduction of explanations significantly suppressed this tendency.
  3. Improvements in subjective trust and understanding of AI were primarily associated with the presence of explanations, but were unrelated to group composition.
  4. The absence of explanations led groups to perceive AI recommendations as authoritative, potentially amplifying cognitive biases (e.g., anchoring effects and diffusion of responsibility).

• Advantages:
  Compared to existing research, this experiment not only validates the role of XAI but also reveals the amplifying effect of explanations in group decision-making. It presents an important insight: explanations are not merely tools for enhancing transparency but can effectively mitigate group biases and foster deeper decision-making reflection.

• Experiment and Evaluation:
  The experiment demonstrated that under conditions with explanations, the rate of over-reliance in group decision-making was lower than in individual decision-making. Conversely, under conditions without explanations, groups exhibited significantly higher dependence on misleading AI recommendations.

• Limitations and Future Directions:

  1. Most participants in the experiment were non-experts, meaning the findings may not directly generalize to high-complexity or high-risk scenarios such as healthcare or judicial domains.
  2. The current study only involved two-person groups, leaving the decision behaviors of larger groups unexplored.
  3. Different types of explanations (e.g., counterfactual or example-based explanations) were not tested, necessitating future research to evaluate the effects of various explanation methods.

Conclusion

The paper effectively demonstrates that in AI-assisted decision-making scenarios, XAI can significantly improve the quality of group decisions and reduce group over-reliance on incorrect AI recommendations. This study represents an important step in understanding the impact of group dynamics on AI-assisted decision-making and how explainability techniques can optimize human-AI collaboration. It provides theoretical foundations and guidance for the future design of AI tools centered around group decision-making.