The Unintended Costs of Platform Interventions: Black-Owned Restaurants and Yelp Reputation

Research Background and Issues

• Identified Issues: The author investigated the impact of adding the "Black-owned" attribute on the Yelp platform and revealed potential unintended costs and harms of this design intervention, including a decline in star ratings for Black-owned restaurants and bias favoring non-Black users on the platform.
• Significance: Design interventions on digital platforms are often aimed at increasing visibility, but for marginalized groups, increased exposure may lead to unintended negative consequences. In the context of addressing racial inequality, understanding the effects and risks of such designs is crucial, especially as digital platforms increasingly become central to commercial and social interactions.
• Research Motivation and Related Work: Drawing on critical theories related to platform capitalism, racial capitalism, and "predatory inclusion," the author highlights how interventions like those on Yelp may reinforce racial awareness while triggering implicit or explicit racial biases, ultimately harming the targeted group.

Solution

• Proposed Method:

  • Collected over 250,000 Yelp reviews for Black-owned and non-Black-owned restaurants in Detroit and Los Angeles.
  • Used statistical and trend analysis methods to explore whether the "Black-owned" attribute influenced the number of reviews and star ratings.
  • Analyzed the intervention effects using Critical Race Theory and the Design Justice framework.

• Innovative Aspects of the Solution:

  • Combined quantitative analysis with qualitative content exploration to identify reviews mentioning "Black-owned" and examine significant correlations between these mentions and rating data.
  • Applied perspectives of platform capitalism and racial capitalism to deepen the understanding of the impacts of digital platform design interventions.
  • Proposed an evaluation approach centered on affected communities, aiming to redefine the outcomes of platform design interventions from the perspective of design justice.

• Implementation Steps and Key Techniques:

  • Data Collection: Used Python tools to scrape review data from Yelp API and platform pages.
  • Data Cleaning: Processed duplicate data, standardized dates, and removed special characters.
  • Analysis Methods: Conducted statistical tests (t-tests and proportion tests) and visualized trend analyses.
  • Critical Analysis: Incorporated concepts like "predatory inclusion" and platform visibility into data analysis to examine the social impacts of the intervention.

Research Findings

• Specific Findings:

  • The addition of the "Black-owned" attribute significantly increased the proportion of reviews mentioning Black-owned restaurants, but the average star ratings for these reviews were lower.
  • Comparative analysis showed that the average star ratings for Black-owned restaurants significantly declined one year after the intervention, with a larger drop compared to non-Black-owned restaurants.
  • Although the exposure of Black-owned restaurants on the platform increased, their online reputation on Yelp did not improve sustainably, and the decline in ratings may negatively affect their business performance.

• Advantages Compared to Existing Solutions:

  • The author critically highlighted the dynamics of "implicit racial bias" and "predatory inclusion" in platform intervention design, providing a new perspective for understanding design failures involving marginalized groups.
  • Proposed a design framework emphasizing community engagement and structural challenges rather than relying solely on individual user behavior improvement.

• Experimental or Evaluation Results:

  • Data analysis confirmed that users were more likely to give lower ratings after becoming aware of Black ownership, revealing the limitations of the intervention strategy.
  • The increase in the total number of reviews was temporary, eventually returning to levels consistent with the pandemic period.

• Limitations and Future Directions:

  • Limitations:
    • The study focused on the restaurant industry and did not cover other types of businesses.
    • The research period was limited to the COVID-19 pandemic, which may have influenced the data.
    • Yelp's API restrictions pose technical challenges for future expanded research.
  • Future Directions:
    • Explore the motivations behind user review behavior, particularly discourse patterns in review content.
    • Investigate how design interventions relate to the racial and cultural backgrounds of user groups.
    • Develop deep engagement models with target groups (e.g., Black restaurant owners and users) to propose more actionable design strategies.

Summary and Recommendations

• The author emphasizes that platforms implementing similar identity-based design interventions should deeply understand their target user groups and contexts to avoid shifting the burden of addressing issues onto marginalized communities. Incorporating Critical Race Theory and the Design Justice framework into the design process can help prevent harmful consequences of interventions.
• Future platform designs should go beyond merely increasing visibility and focus on education and anti-discrimination strategies, reallocating resources and knowledge to better meet the needs of marginalized groups.