Contextualizing User Perceptions about Biases for Human-Centered Explainable Artificial Intelligence

Title of the Paper

Contextualizing User Perceptions about Biases for Human-Centered Explainable Artificial Intelligence

Paper Information

• Subject Area: Explainability in Artificial Intelligence (AI) and User Perception of Bias
• Keywords: Artificial Intelligence, Human-Computer Interaction, Explainable AI, Bias, Transparency, User Perception, Human-Centered Computing

Research Background and Issues

• Identified Problems or Challenges:

  1. A lack of understandability when AI systems or their outputs exhibit bias may reduce user trust and hinder adoption.
  2. AI bias may originate during the data collection phase and propagate through modeling and deployment, leading to societal biases (e.g., gender, race).
  3. Users often lack understanding of the algorithmic mechanisms behind AI, especially when faced with the "black box" problem.
  4. The impact of AI bias may vary depending on the application context, with high-risk scenarios such as healthcare or recruitment being particularly sensitive.

• Significance of the Research:

  1. As AI technology becomes more widely applied, understanding how users perceive AI and address its biases is crucial for improving explainability in AI design.
  2. AI ethics related to fairness, accountability, and transparency have become essential pillars for developing trustworthy AI systems.

• Motivation and Related Work:

  1. While existing XAI (Explainable AI) research often focuses on algorithmic transparency and model interpretability, studies on user perception are relatively scarce, particularly for general end-users rather than technical experts.
  2. This study aims to explore how users understand AI bias in social and technological contexts from the perspectives of human-computer and social interaction.

Proposed Solution

• Proposed Methods or Solutions: This study adopts a user-centered approach by conducting in-depth interviews with two key stakeholder groups: general end-users (n=24) and AI engineers (n=15), to explore user perceptions of AI bias and their needs for explanations.

• Innovative Aspects of the Solution:

  1. Emphasizes supplementing existing computation- and algorithm-focused XAI research with insights from user perceptions.
  2. Differentiates and clarifies bias perceptions and explanation needs across different contexts (high-risk vs. low-risk).
  3. Proposes refined design considerations to enhance AI transparency and user trust in the system.

• Implementation Steps and Key Techniques:

  1. Interview Design:
     • Participants include end-users who regularly use AI applications and engineers responsible for AI development.
     • The interview guide is divided into two main parts: general AI usage experiences and six biased AI scenarios (e.g., medical diagnosis, executive recruitment).
  2. Scenario Selection:
     • High-risk application scenarios: such as healthcare, finance, and recruitment systems.
     • Low-risk application scenarios: such as image search, text autocomplete, and language translation.
  3. Data Processing and Analysis:
     • Record and transcribe interview content.
     • Use Atlas.ti software for open coding, categorization, and grounded theory coding.

Research Findings

• Specific Findings:

  1. How users perceive AI bias:
     • Many users view bias as a reflection of societal realities.
     • Some users attribute bias to algorithmic flaws, while others blame societal factors or their own lack of knowledge.
  2. Differences in user explanation needs:
     • Users want systems to transparently display the sources of bias and key decision parameters (e.g., types of data used).
     • In high-risk scenarios (e.g., healthcare), users have a stronger demand for explainability and lower tolerance for bias.
  3. Recommendations for addressing bias:
     • Systems should make data sources and algorithmic parameters more transparent.
     • Introduce peer user evaluations or audits by external trusted organizations.
     • To mitigate information overload, disclosure should be layered appropriately.

• Advantages Compared to Existing Research:

  1. Provides a more nuanced understanding of high-risk and low-risk scenarios.
  2. Bridges the cognitive perspectives of end-users with the technical views of engineers, offering more practical recommendations for user-centered XAI design.

• Experimental or Evaluation Results:

  1. General users tend to rely on personal experience or external trusted institutions to validate AI outputs.
  2. The closer bias is to users' personal boundaries or the more severe the consequences (e.g., life and health), the higher their demand for model transparency.
  3. Engineers focus more on optimizing model performance (accuracy) based on user feedback, with less attention to specific user perception concerns.

• Limitations and Future Directions:

  1. The study is primarily limited to Taiwan (participants were predominantly Chinese-speaking); future research could include cross-cultural comparisons.
  2. The proportion of female engineers recruited was low; future studies could examine the impact of gender on AI bias perception.
  3. Encourages follow-up studies using quantitative methods to expand sample size and validate the generalizability of bias perception findings.