How should AI Systems Talk to users when Collecting their Personal Information? Effects of Role Framing and Self-referencing on Human-AI Interaction

Title of the Paper

How Should AI Systems Talk to Users when Collecting their Personal Information? Effects of Role Framing and Self-Referencing on Human-AI Interaction

Paper Information

• Research Domain: Human-Computer Interaction, AI Privacy and Trust
• Keywords: Human-AI Interaction, Social Cues, AI System Social Presence, Power Users, Privacy Computing, Trust Building

Research Background and Problem

• Identified Issues or Challenges: AI technologies increasingly rely on users' personal information to provide personalized services, but privacy concerns may deter users from sharing their data. This creates a tension between the demand for personalized services and the importance of privacy protection.

• Significance: When designing conversational AI systems, it is crucial to strike a balance between enhancing user experience and promoting informed decision-making. This includes determining ethical and effective communication strategies to build trust while requesting personal information from users.

• Research Motivation and Related Work: Most current AI systems adopt formal, function-oriented explanations, often emphasizing the value of the services provided. However, whether this "service-oriented" approach is sufficient remains uncertain. This raises a core question: should AI systems portray themselves as "help-seekers" or "help-providers"? Previous studies suggest that machines exhibiting social behaviors, such as using self-referencing terms like "I," may enhance their likability and trustworthiness.

Solution

• Proposed Method or Solution: The authors designed four communication styles (help-oriented, request-oriented, combined, and neutral) and explored how AI role framing (help-seeker vs. help-provider) and self-referencing cues (e.g., using "I") affect users' trust and privacy perceptions.

• Innovations:

  1. Directly testing the concepts of help-seeking and help-providing in AI systems and combining them.
  2. Incorporating user familiarity with AI systems (power users vs. general users) as a variable to examine individual differences in trust-building.
  3. Modeling how this interaction design influences users' privacy-related behaviors.

• Implementation Steps and Key Techniques:

  1. Experimental Design: Simulating the news recommendation system Mindz, employing a 2 (presence of "I" cues) × 4 (role framing) experimental design to measure users' trust, privacy concerns, and willingness to disclose personal information.
  2. Application of Social Presence Theory: Testing whether social cues influence users' perceptions by providing a sense of social presence.
  3. Inclusion of Contextual Variables: Considering general privacy concerns and proficiency in using AI systems (Power Usage) as influencing factors.

Research Findings

• Specific Findings:

  1. When the system adopts a "help-seeker" role frame, power users exhibit higher trust levels, while non-power users prefer the combined "help-provider + help-seeker" frame.
  2. Systems using self-referencing cues like "I" reduce social presence and subsequently lower users' trust and willingness to share data.
  3. Power users are more sensitive to the system's social presence, with trust significantly increasing when the role frame is "help-seeker."

• Comparison with Existing Solutions: Current designs typically focus on "data services" or "machine identity disclosure." This study clarifies that merely emphasizing AI capabilities may not always encourage voluntary disclosure of users' private data.

• Experimental or Evaluation Results:

  1. Quantitative Results:
     • Among 294 users, experimental data showed that the "help-seeker" role frame outperformed the "help-provider" frame (especially among power users).
     • Privacy concerns were reduced under both role frames.
  2. Data Interpretation:
     • In privacy-sensitive interactions, excessive system socialization may trigger users' defensive psychology.

• Limitations and Future Directions:

  1. The experiment measured users' behavioral intentions rather than actual data disclosure behaviors.
  2. As a novel system, the study did not account for users' expectations of established branded systems.
  3. The simulated privacy collection scenario was relatively narrow; future research could expand to more sensitive domains such as financial or health information.
  4. The prioritization of "combining two cues" remains unclear, and further validation is needed to enhance explanatory power.

This study explores and identifies the social cue elements required for designing transparent conversational AI systems that balance user trust and data collection needs. It also provides clear directions for future research based on identified limitations.