"Impressively Scary:" Exploring User Perceptions and Reactions to Unraveling Machine Learning Models in Social Media Applications

Research Background and Issues

• What problems or challenges did the authors identify?
  The authors identified that the local machine learning (ML) models used in current social media applications, particularly those for visual analysis (e.g., Instagram and TikTok), lack transparency. Users are unable to understand when, where, and how these models process their data. More specifically, users have little to no understanding of the actual functionality of these models and their potential impact on data, which raises concerns about privacy, security, and trust.

• Why is this issue important?
  With the widespread application of ML technologies and AI algorithms in social media, these technologies can process users' local data, such as images and videos, in real time. In an era of increasing privacy sensitivity, the lack of transparency may lead to user resistance to machine learning and even a loss of trust in social media platforms. Additionally, the lack of transparency prevents users from effectively deciding and managing how their data is used.

• Research Motivation and Related Work
  Previous studies on the transparency of social media algorithms have primarily focused on users' perceptions of content recommendation and advertising algorithms, with less attention given to the real-world functionality of ML models and their actual impact on users. Moreover, while there are studies on ML model introspection (e.g., through patents or simulated models to study user reactions), there is a lack of systematic analysis that presents real-world application models to users. Therefore, this study aims to fill this gap by analyzing users' reactions and behavioral changes after interacting with and understanding these models, exploring opportunities and challenges for improving transparency.

Solution

• What methods or solutions did the authors propose?
  To explore how users perceive the deployment and impact of machine learning models, the authors designed a user study involving 21 participants. These participants experienced data interaction with real TikTok and Instagram ML models to understand the behavior and functionality of these models.

• What is innovative about this solution?
  A significant innovation of this study is the reverse engineering of real ML models used on social media, allowing users to experience the actual operation of these models. This approach provides unprecedented ecological validity for research in this field. Additionally, the study combines semi-structured interviews with short- and long-term behavioral tracking analysis, offering deeper insights into how users genuinely understand and respond to ML models.

• What are the implementation steps and key technologies used?

  1. Participant Recruitment and Screening: An online questionnaire was used to select 21 eligible social media users.
  2. Experiment Design and Process: The experiment consisted of five stages, from assessing users' current understanding of AI/ML to interactive learning activities and direct engagement with TikTok and Instagram models.
  3. Reverse Engineering Models: The authors used dynamic analysis and static decompilation techniques to extract data from locally running ML models in TikTok and Instagram.
  4. User Data Collection and Analysis: Interview data were processed using Reflexive Thematic Analysis (RTA), supported by qualitative data analysis software to code and categorize participant responses into themes.

Research Findings

• What specific findings were obtained?

  1. Limited User Understanding of Current Transparency: Most users believed that AI/ML only processed content they explicitly shared with the platform (e.g., likes, comments, and posts), unaware that data captured by the camera in real time could also be analyzed.
  2. Impact of Model Operation Differences on User Perception: TikTok's model was perceived as "creepy" for continuously analyzing camera data without notification, while Instagram's model, though overly verbose and confusing, was considered more transparent than TikTok.
  3. Diverse Behavioral and Emotional Responses: Eight participants reported significant changes in their social media usage habits within two weeks after the study, with some reducing platform usage or disabling related permissions.

• What advantages does it have compared to existing solutions?
  Unlike previous studies based on patented models or simulated results, this study directly analyzed off-the-shelf ML models on real platforms and allowed users to experience the actual models in a dynamic experimental environment. This real-world context not only deepened participants' understanding but also provided empirical evidence for optimizing social media transparency design.

• What were the experimental or evaluation results?

  1. Users' immediate reactions to model functionality were mostly negative, particularly regarding TikTok's behavior of analyzing user data without notification.
  2. Long-term behavioral changes indicated that negative emotions could translate into reduced platform usage or enhanced privacy protection behaviors.
  3. Users universally demanded greater transparency, but preferences for the level of detail varied; some wanted full disclosure, while others were concerned about information overload or potential negative societal effects.

• Limitations and Future Directions
  The main limitations of the current study include:

  1. Technical Constraints: Due to the complexity of reverse engineering ML models and restrictions from anti-tampering tools and encryption protections, the study focused only on accessible models.
  2. Sample Bias: The participant group primarily consisted of individuals with high educational backgrounds, which may limit the generalizability of the results to the broader social media user base.
     Future directions include:
  3. Exploring how to design user-friendly transparency tools, such as real-time model activity indicators and transparency dashboards.
  4. Investigating the more complex dynamic relationship between transparency and user trust to provide a basis for rationalizing model outputs.
  5. Expanding the study population to include users with lower levels of technological familiarity to more comprehensively test the generalizability of the findings.

This study makes a significant contribution to understanding the relationship between social media transparency and user behavior and proposes a series of highly practical suggestions for future work.