"I know even if you don't tell me": Understanding Users' Privacy Preferences Regarding AI-based Inferences of Sensitive Information for Personalization

Title of the Paper

"I know even if you don’t tell me": Understanding Users’ Privacy Preferences Regarding AI-based Inferences of Sensitive Information for Personalization

Paper Information

• Subject Area: AI Privacy, User Data Personalization, HCI (Human-Computer Interaction)
• Keywords: Personalization, Privacy, Inference, User Consent, Privacy Preferences, Artificial Intelligence, Data Sensitivity, Data Transparency, Experimental Study, Protected Attributes

Research Background and Problem

• Identified Issues or Challenges:

  • Although personalization features enhance user experience by providing recommendations tailored to user preferences, they often rely on collecting user information, which is frequently obtained covertly or without explicit consent.
  • In recent years, platforms have become increasingly precise in AI-based inference of user data, but these inferences often lack transparency, leaving users with limited understanding and potential privacy risks.
  • Particularly, the generation of inferences involving sensitive attributes (e.g., race, nationality, gender) poses risks of discrimination and misinformation.

• Importance of the Problem:

  • The opaque handling of inferred information, especially sensitive data, can have negative consequences for individuals, such as discriminatory practices or erroneous decisions.
  • The widespread application of AI inferences conflicts with the need for user privacy protection, potentially undermining public trust in online platforms.

• Research Motivation and Related Work:

  • Existing literature primarily focuses on users’ retrospective views of inference behaviors (e.g., via privacy dashboards or ad explanations) rather than their decision-making regarding inferences and data usage in real scenarios.
  • This study aims to explore, through experimental research, how users make privacy preference decisions upon perceiving AI-inferred information, providing new directions for improving data usage consent mechanisms.

Solution

• Proposed Method:

  • Two experimental studies (with a total of 877 participants) were conducted to investigate users’ privacy preferences:
    • Experiment 1: Examined user consent for AI-inferred information usage.
    • Experiment 2: Investigated whether users’ attitudes toward explicitly provided information change after learning about AI-based inferences.
  • A simulated personalized recommendation system (Public Arts Recommender, PAR) was designed to collect users’ publicly shared art preferences and generate AI inferences across three sensitivity categories (art preferences, advertising targets, and protected attributes).
  • A four-level data usage consent metric was used to quantify user preferences (ranging from no consent to the broadest usage scope).

• Innovative Contributions:

  • Incorporated AI inference transparency into user privacy decision-making.
  • Investigated the specific impact of inference sensitivity and accuracy on user consent levels.
  • Compared privacy decisions between AI-generated inferred data and explicitly provided user data.

• Implementation Steps and Techniques:

  • Developed a Bayesian network-based user model to generate inferred user attributes from survey data.
  • Categorized privacy-sensitive questions into different classifications (e.g., "public art," "applicable advertising," and "protected attributes").
  • Controlled variables in the experimental design: inference categories and inference accuracy.
  • Data analysis included ANOVA statistical tests on user consent opinions, followed by surveys on privacy attitudes and system perception.

Research Findings

• Specific Results:

  • Users selected stricter data usage consent for highly sensitive inferences (e.g., nationality or race) and were particularly cautious about inaccurate inferences.
  • When the system displayed AI-generated inferences based on user inputs, users significantly reduced their consent for using those inputs.
  • Trust levels in explicitly provided data were higher compared to AI-inferred data.

• Comparative Advantages Over Existing Solutions:

  • This study validated the importance of inference transparency in personalized recommendation systems, providing a theoretical foundation for designing mechanisms that better align with user privacy needs.
  • Introduced the concept of "inference literacy," emphasizing the need to enhance user understanding of AI inference behaviors.

• Experimental or Evaluation Results:

  • On average, the AI inference accuracy in the experiments was approximately 48%, with sensitive inferences (e.g., protected attributes) receiving the highest restrictions in user consent.
  • Inference type (art/advertising/protected attributes) and inference accuracy were significantly correlated with user privacy attitudes.
  • System transparency’s impact on user privacy perception: disclosing sensitive data inferences may improve users’ evaluations of platform data practices.

• Limitations and Future Directions:

  • Methodological limitations include the study’s focus on public art recommendation scenarios and the evaluation’s restriction to a limited set of questions (10 questions).
  • Future research could expand the scope to contextualized scenarios (e.g., health data or social media).
  • Further exploration is needed regarding the privacy impacts of storing inferred data and optimizing consent mechanism designs.
  • Additional studies should investigate how to educate users about AI inference mechanisms to enhance inference literacy.