From Fragmentation to Integration: Exploring the Design Space of AI Agents for Human-as-the-Unit Privacy Management

Paper Title

From Fragmentation to Integration: Exploring the Design Space of AI Agents for Human-as-the-Unit Privacy Management

Publication Info

• Topic area: AI-driven privacy management across digital ecosystems
• Keywords: AI agents, privacy management, human-as-the-unit, cross-boundary privacy, post-sharing tools, user agency, automation, digital footprint, interpersonal privacy, dynamic preferences

Background and Problem

• Problem / challenge: Current privacy tools are fragmented, context-specific, and fail to address users' cross-boundary, evolving privacy needs. Users face overwhelming burdens in managing their digital footprints across applications, platforms, and relationships.
• Significance: Addressing these challenges is critical to empower users with coherent privacy management, reduce cognitive load, and mitigate risks associated with accumulated data and interpersonal privacy boundaries.
• Motivation and related work: Previous research has focused on application-specific privacy controls, emerging technologies, and platform-level interventions, but these approaches overlook the integrated nature of users' digital lives. AI advancements offer potential solutions, yet their application to holistic privacy management remains underexplored.

Solution

• Proposed approach: Development of AI agents for "human-as-the-unit" privacy management, focusing on cross-boundary, post-sharing, and dynamic privacy needs.
• Novelty:
  1. Introduction of nine AI agent design concepts addressing cross-application, temporal, and interpersonal privacy challenges.
  2. Identification of user preferences for post-sharing tools and automated solutions.
  3. Exploration of design factors balancing timing (pre-sharing vs. post-sharing) and user agency (user-controlled, half-autonomous, fully-autonomous).
  4. Empirical validation of user needs and preferences through interviews and speed-dating surveys.
• Procedure and key techniques:
  • Conducted 12 semi-structured interviews to identify privacy challenges and ad-hoc strategies.
  • Developed nine AI agent concepts based on user needs and evaluated them with 116 participants using speed-dating surveys.
  • Ranked concepts using the Plackett-Luce method and analyzed qualitative feedback to assess user acceptance and skepticism.

Results

• Concrete findings:
  • Top-ranked concepts: Digital Identity Manager (half-autonomous), Dynamic Privacy Preference Agent (fully-autonomous), History Sweeper (fully-autonomous).
  • Median relatability and effectiveness scores of 4–5 out of 5 for most concepts.
  • Users preferred post-sharing tools for managing accumulated data and evolving preferences.
• Advantage over baselines: AI agents were perceived as more accurate, efficient, and capable of handling nuanced privacy preferences compared to manual strategies and existing tools.
• Experiments / evaluation:
  • Interviews identified nine key privacy challenges across applications, temporal contexts, and relationships.
  • Speed-dating surveys validated user needs and ranked design concepts based on relevance and effectiveness.
  • Participants valued automation, centralized management, and proactive privacy monitoring.
• Limitations and future work:
  • Focused on mobile contexts; findings may not generalize to other devices or ecosystems.
  • Sample skewed toward Western, technologically proficient populations.
  • Future research should explore non-AI approaches, regulatory mechanisms, and architectures balancing agent capability with security.

Summary

This study introduces AI agents for human-as-the-unit privacy management, addressing fragmented, cross-boundary privacy challenges. Through interviews and surveys, researchers identified user preferences for post-sharing tools and automated solutions that reduce cognitive load and enhance privacy workflows. The findings highlight the promise of AI agents in managing accumulated data and evolving preferences, while emphasizing the need for safeguards against risks like overtrust, centralized vulnerabilities, and erosion of user agency. Future work should refine dynamic preference learning, ensure secure architectures, and integrate regulatory frameworks to responsibly deploy AI-mediated privacy solutions.