Surfacing Problematic Recommender System Behaviors Affecting Music Discoverability: A Think-Aloud Protocol

Paper Title

Surfacing Problematic Recommender System Behaviors Affecting Music Discoverability: A Think-Aloud Protocol

Publication Info

• Topic area: User perceptions of algorithmic harms in music recommender systems
• Keywords: Music discoverability, recommender systems, algorithmic bias, think-aloud protocol, user perceptions, algorithmic auditing, societal biases, commercial imperatives, confinement, music streaming

Background and Problem

• Problem / challenge: Existing research on music recommender systems has focused on technical evaluation, overlooking how users perceive and interpret problematic behaviors such as bias, lack of diversity, and limited discoverability.
• Significance: Understanding user perceptions of algorithmic harms is critical for designing equitable and participatory recommender systems that enhance music discoverability and address cultural and societal impacts.
• Motivation and related work: Prior studies have documented issues like popularity bias, demographic bias, and lack of transparency in recommender systems. However, the lived experiences and folk theories of everyday users remain underexplored, particularly in the context of music streaming platforms.

Solution

• Proposed approach: A qualitative study using the Think-Aloud Protocol (TAP) to capture real-time user perceptions of problematic behaviors in music recommender systems.
• Novelty:
  1. Empirical insights into how digital-native listeners perceive and interpret algorithmic harms in music discovery.
  2. Application of TAP as a user-driven auditing method in the music sector.
  3. Identification of key themes: societal biases, commercial imperatives, and confinement in discovery.
  4. Design implications for participatory and equitable recommender systems.
• Procedure and key techniques:
  • Conducted TAP interviews with 20 Italian young adults during discovery-oriented tasks on music streaming platforms.
  • Thematic analysis combining deductive and inductive coding to identify user perceptions and folk theories of algorithmic behavior.
  • Tasks included identifying prominent, novel, and diverse artists, reflecting on recommender influence, and discussing broader algorithmic harms.

Results

• Concrete findings:
  • Participants identified three key problematic behaviors: reinforcement of societal biases (e.g., gender and ethnic disparities), commercial imperatives driving exposure (e.g., popularity bias and virality), and confinement within narrow niches (e.g., repetitive recommendations).
  • Users developed folk theories attributing these issues to algorithm design, industry dynamics, and personal listening habits.
  • Participants expressed frustration with limited agency to influence recommendations and highlighted the opacity of algorithmic processes.
• Advantage over baselines: Unlike prior studies focusing on technical metrics, this study provides user-centered insights into the real-world impact of recommender systems on music discoverability.
• Experiments / evaluation:
  • Sample: 20 Italian young adults (11 women, 9 men, 1 non-binary), aged 18–25.
  • Platforms: Predominantly Spotify, with some use of Apple Music, Amazon Music, and YouTube Music.
  • Metrics: Qualitative themes derived from TAP interviews and thematic analysis.
• Limitations and future work:
  • Limited generalizability due to the homogeneous sample (Italian young adults).
  • Reliance on self-reported data and folk theories, which may not reflect actual algorithmic mechanisms.
  • Future work will include ecological momentary assessment and prototype development for participatory auditing tools.

Summary

This study explores how Italian young adults perceive problematic behaviors in music recommender systems, focusing on societal biases, commercial imperatives, and confinement in discovery. Using the Think-Aloud Protocol, participants articulated folk theories and frustrations with algorithmic opacity and limited agency. The findings highlight the need for participatory auditing tools and more equitable recommender systems that balance user preferences with diversity and exploration. Future research will expand on these insights by developing prototypes and incorporating naturalistic listening practices.