Hesitation and Tolerance in Recommender Systems

Paper Title

Hesitation and Tolerance in Recommender Systems

Publication Info

• Topic area: User interaction states in recommender systems and their impact on engagement and retention.
• Keywords: hesitation, tolerance, recommender systems, user retention, engagement, user behavior, cognitive load, trust, feedback signals.

Background and Problem

• Problem / challenge: Current recommender systems rely heavily on coarse metrics like clicks and dwell time, which fail to capture nuanced user states such as hesitation and tolerance. These metrics often misinterpret user dissatisfaction or wasted effort as positive engagement.
• Significance: Misinterpreting user behavior leads to reduced trust, frustration, and attrition, undermining long-term user retention and satisfaction.
• Motivation and related work: Prior research has focused on user behavior modeling and optimization objectives like click-through rate (CTR) and dwell time but has overlooked intermediate states like hesitation and tolerance. This paper aims to fill this gap by systematically identifying and modeling these states.

Solution

• Proposed approach: Introduce and operationalize hesitation and tolerance as measurable intermediate interaction states in recommender systems.
• Novelty:
  1. Conceptualizing hesitation and tolerance as distinct states between interest and disinterest.
  2. Empirically establishing their prevalence and impact on user retention through surveys and behavioral analyses.
  3. Proposing modeling strategies that incorporate tolerance signals as weak positives or negatives.
  4. Demonstrating practical improvements in retention through online A/B testing.
• Procedure and key techniques:
  1. Conducted two large-scale surveys (N = 6,644 and N = 3,864) to identify hesitation and tolerance states and their consequences.
  2. Analyzed behavioral logs from e-commerce (Taobao) and short-video (Kuaishou) platforms to correlate tolerance with reduced engagement.
  3. Performed large-scale online A/B tests on a video platform to validate the impact of incorporating tolerance signals into recommendation models.

Results

• Concrete findings:
  • 94% of survey respondents reported experiencing hesitation, and 59% reported frustration due to tolerance.
  • Behavioral analysis showed that higher tolerance correlates with reduced engagement on both e-commerce and video platforms.
  • Online A/B tests demonstrated up to a 0.67% increase in next-day retention by incorporating tolerance signals.
• Advantage over baselines:
  • Improved retention rates with minimal engineering changes.
  • Enhanced user satisfaction by reducing wasted effort and frustration.
• Experiments / evaluation:
  • Surveys used scenario-based questions to elicit user experiences with hesitation and tolerance.
  • Behavioral datasets from Taobao and Kuaishou were analyzed for tolerance signals (e.g., clicks without purchases, shallow video viewing).
  • Online A/B tests evaluated two strategies: treating tolerance as negatives and as weak positives, showing measurable retention gains.
• Limitations and future work:
  • Limited to e-commerce and short-video platforms; generalizability to other domains (e.g., news, live streaming) is uncertain.
  • Surveys recruited from developer communities, potentially biasing the sample.
  • Focused on short-term retention metrics; longer-term impacts and subjective measures like trust remain unexplored.

Summary

This study introduces hesitation and tolerance as critical intermediate states in user-recommender interactions, highlighting their prevalence and negative impact on user satisfaction and retention. Through surveys, behavioral analyses, and online experiments, the authors demonstrate that modeling tolerance as a distinct feedback signal improves next-day retention by up to 0.67%. These findings challenge traditional reliance on metrics like clicks and dwell time, advocating for more nuanced, user-centered evaluation frameworks. Future work should explore broader domains, longer-term outcomes, and ethical considerations in modeling user emotions.