What's Privacy Good for? Measuring Privacy as a Shield from Harms due to AI Inference of Personal Data

Paper Title

What's Privacy Good for? Measuring Privacy as a Shield from Harms due to AI Inference of Personal Data

Publication Info

• Topic area: Privacy and AI-based decision-making in education and employment contexts.
• Keywords: Privacy, AI inference, harm-centric privacy, procedural justice, education, employment, personal data, discrimination, fairness, inclusivity.

Background and Problem

• Problem / challenge: Existing privacy frameworks fail to capture harms arising from AI inference of personal data, especially when data is not inherently private or sensitive. Current approaches inadequately address privacy violations from inferred data and established norms.
• Significance: Privacy violations from AI inference can lead to significant harms, such as discrimination, manipulation, and stereotyping, affecting individuals' autonomy and fairness in high-stakes contexts like education and employment.
• Motivation and related work: Prior frameworks, such as privacy as concealment or control, and contextual integrity (CI), have limitations in addressing harms from inferred data or established practices. A harm-centric framing, focusing on negative effects of data use, has been proposed but not formally operationalized or empirically validated.

Solution

• Proposed approach: Conceptualize privacy as a shield against perceived harms from personal data use, operationalized through a harm-centric framework in the contexts of AI-based decision-making in education and employment.
• Novelty:
  1. Formalization of a harm-centric privacy framework to measure perceived privacy.
  2. Empirical validation of the framework's reliability and consistency across contexts and demographics.
  3. Identification of nuanced harm perceptions across population segments, advancing equitable privacy.
• Procedure and key techniques:
  • Conducted an online study with 400 US college and university students.
  • Measured perceptions of 14 privacy harms across six personal data types (e.g., demographics, personality traits) in education and employment contexts.
  • Statistical analyses (Cronbach’s alpha, factor analysis, confirmatory factor analysis) validated the reliability and invariance of the harm-centric framework.
  • Explored variations in harm perceptions across demographics and contexts using mixed linear models and Mann-Whitney U tests.

Results

• Concrete findings:
  • High reliability of the harm-centric framework (Cronbach’s alpha > 0.93) across contexts and data types.
  • Privacy harm perception was invariant across demographic groups and contexts, suggesting broad applicability.
  • Demographic, emotional state, and disability data were perceived as most harmful, while motivation and creativity were less concerning.
  • Female and older participants expressed more concerns about privacy harms compared to male and younger participants.
• Advantage over baselines:
  • Captures privacy harms from inferred data and established practices, which are missed by prior frameworks like CI.
  • Surfaces nuanced harm perceptions, enabling targeted mitigation for vulnerable groups.
• Experiments / evaluation:
  • Surveyed 400 participants using Likert-scale harm statements.
  • Analyzed responses using statistical methods to validate reliability, invariance, and demographic variations.
• Limitations and future work:
  • Sample bias due to online recruitment; results may not generalize to the entire US student population.
  • Focused on six data types and two contexts; future studies could expand to other domains and harms.
  • Did not investigate harms from mere surveillance or non-data-driven privacy violations.

Summary

This paper introduces a harm-centric conceptualization of privacy, operationalized to measure perceived harms from AI inference of personal data in education and employment contexts. The framework was empirically validated, demonstrating reliability, invariance across demographics, and nuanced harm perceptions. Key findings include heightened concerns about demographic, emotional state, and disability data, and significant variations in harm perceptions across gender, age, and race. The approach advances equitable privacy by identifying vulnerable groups and providing actionable insights for mitigating harms. Future work could extend this framework to other domains and contexts.