Who Is Included in Human Perceptions of AI?: Trust and Perceived Fairness around Healthcare AI and Cultural Mistrust

Title of the Paper

Who Is Included in Human Perceptions of AI?: Trust and Perceived Fairness around Healthcare AI and Cultural Mistrust

Bibliographic Information

• Topic Area: Trust in AI algorithms, healthcare fairness, and cultural mistrust
• Keywords: AI perception, fairness, trust, healthcare AI, Group-Based Medical Mistrust Scale, Black perspectives
• Conference: CHI Conference on Human Factors in Computing Systems (CHI 2021)
• Publication Year: 2021
• Authors: Min Kyung Lee, Kate Rich

Research Background and Problem

• Identified Issues or Challenges:

  • The widespread adoption of AI in fields such as healthcare, education, and justice has heightened concerns about trust in algorithmic decision-making.
  • Existing research shows that trust in algorithmic decisions is significantly lower than trust in human decisions, especially for tasks requiring subjective judgment or attention to individual uniqueness.
  • However, trust levels in "human decision-makers" vary across different groups, particularly among Black Americans, whose historical experiences of systemic racial discrimination have fostered mistrust in human systems.

• Significance:

  • Understanding specific social groups' perceptions of AI decision-making can help mitigate inequalities in healthcare.
  • AI in healthcare impacts patient experiences and has the potential to exacerbate existing disparities.

• Motivation and Related Work:

  • Current research on trust in AI often neglects cultural and social context differences.
  • This study focuses on AI decision-making in diagnosing skin cancer, comparing perceptions of trust and fairness between individuals with high and low levels of medical mistrust.
  • The goal is to explore the role of cultural mistrust in healthcare AI scenarios and provide data to support future research on social group perceptions of algorithmic fairness.

Solution

• Research Methods:

  1. Online Experiment:
     • Conducted experiments to compare trust in AI and human doctor decisions in medical contexts.
     • Measured participants' mistrust using the Group-Based Medical Mistrust Scale (GBMMS) and divided them into high and low mistrust groups.
     • Presented scenarios of AI diagnosing skin cancer versus human doctors and evaluated perceptions of fairness and trust.
  2. Interview Study:
     • Conducted semi-structured interviews to gather perceptions of healthcare AI among high-mistrust individuals and identify information that could enhance trust in AI.

• Research Hypotheses:

  • H1: Individuals with low mistrust in human systems will perceive AI decisions as less trustworthy and fair compared to human doctor decisions.
  • H2: Individuals with high mistrust in human systems will perceive AI decisions as equally or more trustworthy and fair compared to human doctor decisions.

• Implementation Steps:

  • Recruited Black and White participants for the online experiment, collecting data through two rounds of surveys (first round measured GBMMS scores, second round assessed perceptions).
  • During interviews, presented various descriptions of healthcare AI, including basic descriptions, data-driven descriptions, fairness-driven, and anti-discrimination-driven descriptions.

Research Findings

• Experimental Results:

  • Low Mistrust Group: Human doctor decisions were perceived as fairer and more trustworthy, consistent with previous literature.
  • High Mistrust Group: Trust and fairness ratings for AI decisions and human doctor decisions showed no significant differences.
  • Group Differences:
    • Among the high mistrust group, Black participants rated trust and fairness significantly lower than White participants, with overall higher medical mistrust (GBMMS scores) among Black participants.

• Interview Results:

  • Distrust in healthcare AI was closely tied to individuals' negative experiences with human doctors, particularly concerns about discrimination.
  • Views on AI fairness and bias: Some high-mistrust participants questioned AI's potential biases, doubting its ability to treat Black patients fairly compared to traditional systems.
  • Desired additional information: Participants expressed a need for more detailed information about AI transparency (e.g., representativeness of training data, privacy protection, accuracy).
  • Responses to AI descriptions:
    • Data-driven descriptions were generally perceived as more trustworthy.
    • High-mistrust participants paid attention to fairness statements, prioritizing anti-discrimination descriptions but remained skeptical about their actual effectiveness.

• Contributions and Significance:

  • Highlighted the neglect of social and cultural differences in current research on algorithmic trust.
  • Provided new insights into effective design strategies for high-mistrust groups and low-trust algorithms, such as offering more transparent and data-rich presentations.

• Limitations and Future Directions:

  • The study relied on online experiments and interviews with a relatively small sample size.
  • Did not test multiple healthcare scenarios or include broader racial and gender groups, such as transgender or non-binary individuals.
  • Future research could expand into real-world healthcare settings and explore perceptions among diverse audiences, including Latinx and Asian communities.

Conclusion

This study examined the impact of mistrust in human decision-making systems among Black individuals with high medical mistrust on perceptions of fairness and trust in healthcare AI. It revealed the importance of cultural differences and emphasized the need to focus on transparency and fairness-driven presentations in designing trustworthy algorithms. The findings provide recommendations and directions for reducing social group inequalities in healthcare.