Designing Privacy Choice in Generative AI Chatbot Ecosystems

Paper Title

Designing Privacy Choice in Generative AI Chatbot Ecosystems

Publication Info

• Topic area: Privacy choice design in Generative AI ecosystems
• Keywords: Generative AI, privacy choice, chatbot ecosystems, personalization, data sharing, first-party ecosystems, third-party ecosystems, user trust, privacy control, transparency

Background and Problem

• Problem / challenge: Existing Generative AI ecosystems lack meaningful interfaces for users to make informed privacy choices. Current solutions focus on system-level protections rather than user-centric privacy controls.
• Significance: The shift of GenAI from standalone tools to interconnected ecosystems amplifies privacy risks due to complex data flows, making user empowerment in privacy decisions crucial.
• Motivation and related work: Prior research has explored privacy notices and controls in traditional systems but has not adequately addressed the unique challenges posed by GenAI ecosystems, such as opaque and dynamic data exchanges. This paper aims to fill this gap by investigating privacy choice design in GenAI ecosystems.

Solution

• Proposed approach: Investigating privacy choice design in GenAI ecosystems through vignette-based surveys and interviews, focusing on two scenarios: upstream personalization and downstream data sharing.
• Novelty:
  1. Scoped investigation of user perceptions, expectations, and concerns regarding privacy choice interfaces in GenAI ecosystems.
  2. Identification of how ecosystem type and timing of privacy choices influence user trust, control, and willingness to engage.
  3. Exploration of paradoxical user perceptions of first-party vs. third-party ecosystems.
  4. Design implications for creating usable and trustworthy privacy choice interfaces.
• Procedure and key techniques:
  • Conducted two vignette-based surveys with 486 participants and follow-up interviews with 16 participants.
  • Examined two scenarios: upstream personalization (user data input to GenAI) and downstream data sharing (GenAI data output to external providers).
  • Analyzed the effects of privacy choice timing (at-setup vs. just-in-time) and ecosystem type (first-party vs. third-party) on user perceptions.

Results

• Concrete findings:
  • At-setup privacy choices provided a stronger sense of control but were sometimes perceived as intrusive.
  • Just-in-time choices offered contextual relevance but were seen as disruptive and raised suspicion.
  • First-party ecosystems (e.g., Google Gemini) were trusted for accountability but raised concerns about ecosystem-wide data harvesting.
  • Third-party ecosystems (e.g., ChatGPT) were preferred for personalization but required greater transparency about data sharing.
• Advantage over baselines:
  • Identified nuanced user preferences and paradoxical trust patterns not captured in prior research.
  • Provided actionable insights for balancing anticipatory and contextual privacy controls in GenAI ecosystems.
• Experiments / evaluation:
  • Surveys measured perceived privacy control, trust, and willingness to use GenAI systems across 984 vignette responses.
  • Interviews provided qualitative insights into user reasoning and preferences.
• Limitations and future work:
  • Limited to U.S.-based participants; cultural and legislative differences were not explored.
  • Focused on timing and ecosystem type; other design dimensions (e.g., modality, granularity) remain unexplored.
  • Vignette-based approach simplified real-world interactions; future studies should examine dynamic, multi-turn GenAI conversations.

Summary

This paper investigates privacy choice design in Generative AI ecosystems, focusing on how timing (at-setup vs. just-in-time) and ecosystem type (first-party vs. third-party) influence user perceptions. Surveys and interviews reveal paradoxical trust patterns: users trusted third-party ecosystems for personalization but preferred first-party ecosystems for data sharing. At-setup choices provided anticipatory control, while just-in-time choices offered task-specific clarity but disrupted interaction flow. The findings highlight the need for balanced privacy interfaces that combine anticipatory and contextual controls, enhance transparency, and support minimal, task-oriented data use. These insights inform the design of user-centric privacy solutions in GenAI ecosystems.