In Suspense About Suspensions? The Relative Effectiveness of Suspension Durations on a Popular Social Platform

Research Background and Issues

• Issues or Challenges: Many digital platforms currently impose consequences that influence user behavior (e.g., warnings, temporary or permanent account bans) to address violations of community rules. However, compared to the criminal justice field, research on the relative effectiveness of these consequences in digital platforms—particularly regarding the impact of different suspension durations—is limited and lacks robust evidence. For instance, whether longer suspension durations are more effective than shorter ones in improving behavior remains unclear due to the absence of empirical studies. Additionally, the long-term effectiveness of these interventions and whether they vary across different user groups have not been thoroughly explored.
• Significance: As platform policies shift from punitive measures to a focus on restorative justice (e.g., strategies aimed at promoting behavioral improvement), understanding the relative impact of suspension durations on user behavior can help platforms enhance rule enforcement and reduce repeat violations. Such optimization not only aids in minimizing harmful behaviors but also mitigates the negative impact of user churn or reduced engagement on business metrics.
• Research Objectives: This paper primarily addresses the following three research questions:
  1. What is the impact of suspension duration on subsequent violations and platform engagement (e.g., active days) among violators?
  2. Does the effectiveness of suspension duration diminish over time? If so, what is the timeframe and extent of this decline?
  3. How do different suspension durations affect first-time violators compared to repeat violators?

Solution

• Experimental Design:
  • Two large-scale field experiments were conducted, involving experimental data from over 770,000 users, targeting first-time violators and repeat violators respectively.
  • Experiment 1 compared suspension durations of 1 hour and 1 day, applied to first-time violators.
  • Experiment 2 compared suspension durations of 1 day and 3 days, applied to repeat violators.
  • Both experiments utilized the CUPED method to control for pre-experiment differences and enhance the sensitivity of experimental results.
• Key Techniques:
  • The platform determined user violations through a combination of user reports, automated classifiers, and manual review.
  • During the experimental window, metrics such as repeat violations, the number of reports received, and time spent on the platform were recorded.
• Innovations:
  1. This study is the first to comprehensively compare the policy effects of different suspension durations on digital platforms, providing precise data support.
  2. Multiple behavioral and engagement metrics (e.g., repeat violation rate, total number of penalties, user active days) were employed to measure the overall impact of suspensions.
  3. The heterogeneous effects of suspension durations on different user groups (first-time vs. repeat violators) were explored.

Research Findings

• Key Findings:

  1. Longer suspension durations (e.g., 1 day vs. 1 hour, 3 days vs. 1 day) were more effective in reducing repeat violation rates (a reduction of 7%-8%) and extending the "pause time" in violation behavior (an increase of 7%-9%).
  2. Compared to the reduction in violations, suspension durations had a smaller impact on overall user engagement (e.g., total time or active days), with insignificant or minimal reductions of 1%-2%.
  3. The effectiveness of suspension durations diminished over time but persisted for at least 3 weeks. For example, after 3 weeks, the repeat violation rate for the 1-day suspension group was still approximately 3% lower than that of the 1-hour suspension group.
  4. For first-time violators, longer suspension durations had a more pronounced effect, reducing violation rates by up to 13%, whereas the reduction for repeat violators was smaller (approximately 4%).

• Advantages:

  • Provides detailed and data-rich empirical research, helping platforms strike a balance between rule enforcement and user engagement.
  • The findings support platforms in implementing more targeted suspension strategies based on users' violation histories.
  • Offers insights for other social and gaming platforms in designing restorative consequence measures.

• Experimental Data:

  • In Experiment 1, the repeat violation rate for users suspended for 1 hour was 27%, while the rate for those suspended for 1 day was 25.2%.
  • In Experiment 2, the repeat violation rate for users suspended for 1 day was 47.2%, while the rate for those suspended for 3 days was 43.4%.

• Limitations and Future Directions:

  1. This study's suspension duration upper limit was 3 days; future research could explore longer suspension durations to assess long-term effectiveness.
  2. The characteristics of the Roblox platform (e.g., ease of creating alternative accounts, age distribution of user groups) may affect the external applicability of the findings.
  3. The experimental design did not directly control or capture potential network interference among users, such as secondary effects arising from discussions about suspension policies among friends.

Conclusion

The study demonstrates that longer suspension durations significantly enhance platform management and promote user behavior improvement, particularly for first-time violators. Although the effects diminish over time, the experiments provide quantitative and empirical evidence to guide platform managers in determining the optimal suspension duration for policy formulation.