Analyzing User Engagement with TikTok's Short Format Video Recommendations using Data Donations

Title of the Paper

Analyzing User Engagement with TikTok’s Short Format Video Recommendations using Data Donations

Paper Information

• Subject Area: Short video content recommendation algorithms and user behavior analysis
• Keywords: TikTok, recommendation algorithms, user engagement, data donation, data privacy, short video platforms, user behavior, social media, data mining

Research Background and Issues

• What problems or challenges did the authors identify?

  • The rising popularity of short videos on social media contrasts with the lack of large-scale empirical studies on how users consume this content and how recommendation algorithms impact user experience.
  • While media reports suggest that TikTok's algorithm accurately recommends videos of interest to users, it may also lead users to problematic content.
  • Existing research often relies on automated accounts or publicly scraped data, lacking insights from real user behavior.

• Why is this issue important?

  • Platforms like TikTok have become mainstream social media content consumption channels, with over 1.3 billion global users. Their recommendation algorithms significantly influence user attention and engagement, with implications for mental health and tech ethics.
  • The optimization of short video platform algorithms for user retention and platform revenue may lead to addictive behaviors and potential risks.

• Research Motivation and Related Work

  • This study addresses the gap in analyzing the effects of short video recommendation algorithms based on real user behavior data.
  • It explores the use of data donation systems to collect data from authentic users, enabling more accurate audits of short video algorithms.

Solutions

• What methods or solutions did the authors propose?

  • The authors designed and implemented a data donation system called "Social Media Donator (SMD)" that allows users to anonymously donate their TikTok viewing history and related data.
  • By analyzing 9.2 million viewing records from 347 users, the study evaluated the effectiveness of TikTok's recommendation algorithm, focusing on metrics such as viewing time, video attention, and engagement (e.g., likes).

• What is innovative about this solution?

  • The data source leveraged users' GDPR rights to access their data, enabling analysis based on real user behavior and avoiding biases from automated account simulations.
  • The system integrated anonymization and customization mechanisms to enhance user trust and privacy protection.

• Implementation Steps

  1. Data Collection and Donation: Users were guided to request and download their activity data packages from TikTok, followed by anonymization and customization processes.
  2. Participant Recruitment and Incentive Mechanism: Participants were recruited via Facebook ads and Twitter outreach, with monetary compensation of $5–16 offered as an incentive.
  3. Data Analysis:
     • Controlled experiments were conducted to infer user viewing durations.
     • Data analysis was performed across dimensions such as video view counts, user engagement behaviors (e.g., like rates and completion rates), and content sources (followed vs. unfollowed accounts).
  4. Sensitivity Analysis: Robustness checks were conducted on active user days and threshold metrics.

Research Findings

• Specific Findings

  1. User Behavior Trends:
     • The number of videos watched and average usage time increased significantly over time. For example, video views doubled after 80 days of use.
     • On average, users consumed approximately 89.9 videos daily, spending 27 to 48 minutes per day.
  2. User Attention Stability:
     • Despite increased viewing time, users' "attention" to videos (the proportion of videos watched to completion) remained stable, averaging 45%.
     • Videos from unfollowed accounts had higher view rates than those from followed accounts.
  3. User Interaction Patterns:
     • Like behaviors increased over time. For instance, the like rate for videos uploaded by followed accounts doubled after 120 days.
     • Users were more inclined to like content from followed accounts, reflecting emotional connections within social networks.
  4. Content Characteristics Analysis:
     • Comparisons revealed that videos from unfollowed accounts were more popular across the platform, with higher view counts, possibly due to more engaging content quality.

• Advantages Compared to Existing Solutions

  • The study introduced a mechanism based on real user data donations, making the analysis more authentic and comprehensive.
  • The anonymization strategy and privacy protection design significantly enhanced the credibility and compliance of user behavior data analysis.

• Experimental and Evaluation Results

  • User engagement (likes) increased significantly over time, while user attention to recommended videos remained stable.
  • The platform's algorithm tended to recommend content from "unfollowed accounts" to enhance content appeal and user retention.

• Limitations and Future Directions

  1. Participant Sample: Participants were primarily concentrated in specific geographic regions or demographic groups, potentially limiting representation of TikTok's overall user ecosystem.
  2. Data Bias: Some fields (e.g., like records) were missing, or platform logging issues limited data fidelity.
  3. Algorithm Inference: While user behavior was analyzed, the study could not directly explain the intentionality of TikTok's algorithm design, such as whether it includes negative reinforcement (e.g., forced exploratory recommendations).
  4. Cross-Platform Comparison: The study focused solely on TikTok, leaving room for future comparisons with other short video platforms like YouTube Shorts.

Conclusion

This paper presents the first empirical analysis of user behavior on short video recommendation platforms based on data donations. The study reveals the complex interaction between users and recommendation algorithms, emphasizing the importance of designing user-friendly and ethically considerate data donation platforms. Furthermore, the research lays the groundwork for auditing short video platform recommendation algorithms and their psychological impacts on users, recommending future focus on platform algorithm transparency and user health management features.