Out-of-Device Privacy Unveiled: Designing and Validating the Out-of-Device Privacy Scale (ODPS)

Title of the Paper

Out-of-Device Privacy Unveiled: Designing and Validating the Out-of-Device Privacy Scale (ODPS)

Paper Information

• Research Areas: User privacy protection, psychometric measurement, privacy threat assessment
• Keywords: out-of-device privacy, out-of-device threats, physical world privacy, psychometric measurement, privacy scale, user behavior prediction

Research Background and Issues

• Problem Identification:

  • Existing privacy scales (e.g., IUIPC) primarily focus on online privacy issues, lacking assessments of privacy threats in the physical world.
  • Users exhibit varying responses to physical world privacy threats, such as shoulder surfing or thermal imaging attacks, and current "one-size-fits-all" protection mechanisms fail to address individual differences.
  • There is a lack of systematic tools to quantify users' prioritization of out-of-device privacy threats in the physical world, which is crucial for designing customized privacy protection mechanisms.

• Significance:

  • As technological devices continuously collect sensitive user data, privacy exposure is no longer limited to online scenarios; protection in physical contexts is equally urgent.
  • Understanding users' prioritization of out-of-device privacy threats can enable the design and prediction of more effective privacy protection solutions, enhancing user experience and mitigating the negative impacts of privacy violations.

• Research Motivation and Related Work:

  • Previous studies have demonstrated that physical threats like shoulder surfing significantly affect user psychology and behavior.
  • While various online privacy protection tools (e.g., visual filters, vibration alerts) exist, there is no unified standard to effectively differentiate users' levels of privacy importance.
  • This study aims to address this research gap by developing and validating a psychometric tool specifically designed to measure users' prioritization of physical world privacy—Out-of-Device Privacy Scale (ODPS).

Solution

• Proposed Method:

  • Develop an 18-item psychometric questionnaire (ODPS) specifically for assessing users' perception of the importance of out-of-device privacy threats in the physical world.
  • Employ an iterative, evidence-based questionnaire development process, including three stages: item generation, questionnaire development, and questionnaire validation.

• Innovations:

  • Define and measure "out-of-device privacy" as an independent privacy dimension, addressing a gap in existing research.
  • Provide a universal and precise tool to extend the concept of privacy from online spaces to physical contexts.
  • Methodologically combine deductive and inductive item development approaches to ensure content representativeness and scientific rigor.

• Implementation Steps:

  1. Item Development:
     • Define "out-of-device privacy" as "the degree to which individuals prioritize protecting data from external threats in the physical world."
     • Generate initial items (67) through literature review and expert interviews.
     • Refine items by removing duplicates, irrelevant, or incomprehensible entries, reducing the list to 31 items.
  2. Questionnaire Development:
     • Conduct small-scale pilot testing on 31 items to assess semantic clarity and response variability.
     • Use exploratory factor analysis (EFA) to identify a single-factor structure from 26 items, ultimately retaining 19 items.
  3. Questionnaire Validation:
     • Validate the questionnaire using large-sample data (N=935) through confirmatory factor analysis (CFA) to confirm model fit indices.
     • Test reliability and validity, including Cronbach's alpha, composite reliability, and convergent validity, refining the scale to 18 items.

Research Findings

• Specific Results:

  • Developed and validated an 18-item psychometric tool, ODPS, to measure users' perception of physical world privacy threats.
  • Established high internal consistency and reliability in large-sample experiments (Cronbach's alpha = 0.917).
  • Demonstrated that ODPS significantly differs from online privacy scales (e.g., IUIPC), with over 85% of total variance unexplained by online privacy concerns, confirming its unique focus on physical world privacy dimensions.

• Comparison with Existing Solutions:

  • Combining ODPS with online privacy scales like IUIPC enables a more comprehensive profile of user privacy preferences, covering both physical and online environments.
  • Enhances the specificity and scientific basis of current privacy research and protection mechanisms in addressing out-of-device threats.

• Experimental or Evaluation Results:

  • Factor analysis confirmed ODPS's single-factor structure and validated its overall reliability and statistical support (CFI=0.923, TLI=0.903, RMSEA=0.066).
  • Correlation analysis with existing privacy scales (e.g., IUIPC and CFIP) indicated that ODPS represents a distinct dimension while maintaining conceptual consistency and differentiation.

• Limitations and Future Directions:

  • Limitations:
    • Samples were exclusively from the UK, potentially introducing regional selection bias.
    • The questionnaire has not fully validated its ability to capture all possible user privacy needs.
  • Future Research:
    • Conduct cross-cultural validation globally to demonstrate ODPS's broad applicability.
    • Explore the potential applications of ODPS in privacy protection mechanism development and study its impact on design.
    • Integrate ODPS with other privacy scales (e.g., IUIPC) to create comprehensive user privacy profiles, offering protection from online to physical world scenarios.