How Much Do Platform Workers Value Reviews? An Experimental Method

Title of the Paper

How Much Do Platform Workers Value Reviews? An Experimental Method

Paper Information

• Subject Area: Reputation systems and algorithmic management in online labor platforms
• Keywords: Online labor platforms, reputation systems, algorithmic management, experimental research, Upwork, behavioral economics, online freelancing, labor market, reputation value, long-term benefits

Research Background and Problem

• Identified Issues or Challenges: Online labor platforms control workers' behavior and job allocation through reputation systems and algorithmic management. However, the quantitative impact of such management, as well as its variation across different platforms and over time, has been largely studied through qualitative analysis.
• Significance: Reputation systems affect workers' job opportunities and earning potential on platforms. Understanding the intensity of such management can improve platform design and working conditions for online laborers.
• Research Motivation and Related Work:
  • Qualitative studies have demonstrated the importance and impact of reputation, particularly on labor platforms like Upwork, Uber, and DoorDash.
  • Reputation systems not only influence the matching of workers and tasks but may also lead to "low-transparency" evaluation and matching mechanisms.
  • Current qualitative literature struggles to compare the intensity of management across platforms and over time, highlighting the potential value of quantitative approaches.

Proposed Solution

• Proposed Method:
  • Design a behavioral experiment to quantitatively assess how much platform workers value reputation by having them choose between monetary bonuses and positive reviews.
  • Use "Willingness to Accept" (WTA) to measure the monetary compensation workers require to forgo a positive review.
• Innovative Aspects:
  • Provides a consistent quantitative method for cross-platform and temporal comparisons of the impact of reputation management.
• Implementation Steps and Key Techniques:
  • Conduct the experiment on the Upwork platform, recruiting over 500 participants.
  • Randomly assign participants to experimental groups with fixed monetary reward options ($25, $50, $75, $125, $175 USD) and record their preferences for monetary rewards or reviews.
  • Estimate demand curves and median WTA using econometric models.
  • Supplement quantitative analysis with qualitative data by analyzing workers' textual feedback to understand their decision-making motivations.

Research Findings

• Specific Results:
  • Among the sample, the median WTA for a single positive review on Upwork was $49 USD.
  • Less experienced workers (below the median experience level) valued positive reviews more highly, with a WTA approximately $30.65 USD higher than that of experienced workers.
  • Qualitative analysis revealed that workers choosing reviews often considered long-term benefits and the role of reputation management, while those opting for monetary rewards were primarily driven by short-term financial needs (especially during the COVID-19 pandemic).
• Advantages Over Existing Solutions:
  • Provides a quantifiable method to assess how much platform workers value reputation, which can be extended to other platforms and time periods.
  • Helps identify specific platform characteristics and market conditions that modulate the intensity of algorithmic management.
• Experimental or Evaluation Results:
  • Even with monetary rewards as high as $175 USD, approximately 32% of workers chose positive reviews, underscoring the importance of reputation systems for platform laborers.
• Limitations and Future Directions:
  • The sample was limited to two categories on the Upwork platform (customer support and graphic design), which may not represent the broader population.
  • The scale of the experiment was constrained by costs, preventing expansion to larger samples.
  • The content of reputation text might influence WTA measurements, necessitating further research on the impact of long-text reviews.
  • Future research could employ other low-cost experimental designs, such as random binding mechanisms or Becker-DeGroot-Marschak models, to reduce costs and expand applicability.
  • The method could be extended to other dimensions of algorithmic management, such as flexibility and task-matching efficiency.