Do Humans Prefer Debiased AI Algorithms? A Case Study in Career Recommendation

Title of the Paper

Do Humans Prefer Debiased AI Algorithms? A Case Study in Career Recommendation

Paper Information

• Domain: Human-Computer Interaction (HCI), Fairness Algorithms, Recommendation Systems
• Keywords: Debiasing Algorithms, Human-Computer Interaction, Career Recommendation, Gender Fairness, AI Fairness, Bias Evaluation, User Acceptance, Behavioral Intervention, Machine Learning Ethics, Socio-Technical Systems

Research Background and Problem

• Issues or Challenges:

  1. Despite rapid advancements in AI fairness research, most efforts focus on algorithmic debiasing while overlooking the complexities of user interactions with fair AI.
  2. Current AI recommendation systems (particularly in career selection) often exhibit gender bias, which may reinforce existing societal gender role stereotypes.
  3. Simply removing bias from AI algorithms may not achieve the desired fairness goals, as human biases can also influence the acceptance of fair systems.

• Significance:

  • Career selection is a critical life decision, directly impacting individuals' socioeconomic status, quality of life, and gender equality in professions.
  • Gender bias not only affects individual career choices but may also exacerbate societal inequalities.

• Motivation and Related Work:

  • Unlike existing studies that focus on developing debiasing algorithms, this research systematically explores how human biases influence interactions with fair AI.
  • Sociological studies have shown that gender stereotypes significantly affect career choices, providing a scientific basis for investigating this issue.

Solution

• Methods or Solutions:

  1. Develop a gender-fair career recommendation system based on a Neural Collaborative Filtering (NCF) model, using machine learning techniques to reduce gender bias in recommendations.
  2. Conduct an online user study using university major recommendations as a case study to explore real users' attitudes toward unbiased systems.
  3. Compare the performance of debiased algorithms and gender-sensitive (biased) recommendation algorithms across dimensions such as user acceptance, gender role consistency, and system usage intention.

• Innovations:

  • Focus not only on algorithmic fairness but also on comprehensively exploring the impact of human biases on interactions with fair AI systems for the first time.
  • Propose a vector projection-based gender bias mitigation algorithm capable of removing biases related to gender variables and their proxies.
  • Combine social psychology methods with algorithmic research, such as incorporating implicit gender bias perception.

• Implementation Steps and Key Techniques:

  1. Algorithm Development:
     • Extract user interest data from the Facebook dataset and build a recommendation model based on Neural Collaborative Filtering (NCF).
     • Use the vector projection method to eliminate explicit and implicit biases related to gender variables.
     • Employ a logistic regression model to output gender-debiased recommendations.
  2. User Study:
     • Design an online user survey to investigate user acceptance of debiased recommendation systems.
     • Use a Generalized Linear Model (GLM) to analyze the relationships between user biases, self-reported career perceptions, and recommendation acceptance.

Research Findings

• Key Findings:

  1. Content analysis shows that the debiased system significantly reduces gender stereotypes in career recommendations, such as equalizing the proportion of computer science recommendations for male and female users.
  2. The debiased recommendation system outperforms the gender-sensitive system in both fairness and accuracy (evaluated using NDCG and UPAR metrics).
  3. User behavior data reveals that despite improvements in algorithmic fairness, users overall preferred recommendations from the gender-sensitive system.

• Advantages Over Existing Methods:

  • Provides a new perspective on AI fairness, extending from "algorithmic fairness" to "social fairness."
  • Proposes novel research directions, such as AI bias explanation and behavioral intervention techniques.

• Experimental or Evaluation Results:

  1. Experimental results indicate:
     • The gender-debiased algorithm achieves the goal of "fairness as accuracy," but user acceptance is lower compared to the biased system.
     • Most participants (both male and female) tend to prefer career recommendations aligned with gender stereotypes, avoiding choices perceived as "dominated by the opposite gender."
  2. User preferences are driven by underlying human biases, and acceptance of fair systems is correlated with perceived gender consistency.

• Limitations and Future Directions:

  • Limitations:
    1. Most participants had already determined their career choices, which may have influenced their openness to the recommendations.
    2. The study only examined gender bias, excluding other protected characteristics such as race or religion.
    3. The study did not fully explore the biases of other stakeholders in career selection, such as employers or admissions officers.
  • Future Directions:
    1. Extend research to high-potential populations (e.g., high school students) to enhance the generalizability of the approach.
    2. Explore scenario transfer, such as fairness recommendation studies incorporating race or disability identities.
    3. Continue developing and validating new frameworks for human-AI collaborative bias mitigation (e.g., joint training of human and AI biases).

The above analysis summarizes the main content and academic contributions of this study, demonstrating how interdisciplinary perspectives can be used to explore AI fairness issues. This provides significant insights for future research in HCI and machine learning.