Auditing E-Commerce Platforms for Algorithmically Curated Vaccine Misinformation

Title of the Paper

Auditing Algorithm-Generated Vaccine Misinformation on E-commerce Platforms

Paper Information

• Subject Area: Credibility of health information and algorithmic bias on e-commerce platforms
• Keywords: Search engines, health misinformation, vaccine misinformation, algorithmic bias, personalization, algorithm auditing, search results, recommendation systems, e-commerce platforms

Research Background and Problem

• Problem Identification:
  • With the growing reliance on internet searches, the quality of vaccine and health-related information has become a critical issue.
  • E-commerce platforms (e.g., Amazon) have been criticized for failing to regulate health-related products and may serve as major conduits for misinformation.
• Significance:
  • Vaccine hesitancy can undermine the goal of herd immunity, especially in the context of the global COVID-19 pandemic.
  • Trust in e-commerce platform recommendations can influence users' purchasing habits and health perspectives.
• Motivation and Related Work:
  • Most research on misinformation in search engines focuses on mainstream search engines, with relatively little attention paid to e-commerce platforms.
  • This paper aims to systematically audit vaccine misinformation on the Amazon platform, uncovering potential issues in its search and recommendation algorithms.

Solution

• Methods and Approach:
  • Two systematic algorithm audits were conducted: non-personalized auditing and personalized auditing.
  • Non-personalized auditing assessed the proportion of misinformation in search results for vaccine-related queries.
  • Personalized auditing explored how user history, through real-world actions such as clicking on products or adding them to the cart, influences search results and recommendations.
• Innovations:
  • The first large-scale systematic audit of vaccine-related misinformation on an e-commerce platform.
  • Development of a comprehensive qualitative annotation framework to label whether health-related products promote misinformation.
• Implementation Steps and Techniques:
  • Annotating the stance of health-related products (promoting misinformation, remaining neutral, or countering misinformation).
  • Comparing "misleading bias" metrics in search and recommendation results by calculating input bias, output bias, and ranking bias to reveal algorithmic issues.
  • Using automated tools (e.g., Selenium and AWS virtual machines) to control experimental variables such as geographic location and user history.

Research Findings

• Specific Results:
  • In the non-personalized audit, 10.47% of search results contained health misinformation, exceeding the proportion of products countering misinformation (8.99%).
  • Search results under default "featured" sorting and "high customer reviews" sorting were biased toward misinformation.
  • The recommendation system exhibited a "filter bubble" effect, where users who purchased or engaged with misinformation products were subsequently recommended more similar products.
• Advantages and Contributions:
  • The first definition of a comprehensive framework for labeling health misinformation on e-commerce platforms.
  • Provided case analyses of how e-commerce platforms amplify health misinformation through their algorithms.
• Experimental Results:
  • When users clicked on or purchased products containing misinformation, the recommendation algorithm further promoted similar products, exacerbating the spread of misinformation.
  • No significant personalization effects were found in autocomplete suggestions.
• Limitations and Future Directions:
  • The experiments did not cover all types of recommendations, such as off-platform email recommendations.
  • The user behavior simulation in personalized auditing was relatively simple and did not fully capture complex real-world scenarios.
  • Future research could expand this method to audit a broader range of topics and study the impact of recommendation systems and search engines on public opinion.

Conclusion and Recommendations

• Key Conclusions:
  • The Amazon platform exhibits significant health misinformation in its search and recommendation systems, including a "vaccine misinformation filter bubble."
  • Current platform algorithms fail to account for the critical importance of health-related issues.
• Recommendations:
  • Short-term strategies: Introduce "bias dashboards" or information interventions in search results (e.g., linking to Wikipedia).
  • Long-term strategies: Modify recommendation algorithms to avoid the concentration of misinformation; implement policies to ban low-quality books or products.
  • Raise public awareness and apply collaborative pressure to push e-commerce platforms to improve their content quality control measures.