Does My Chatbot Have an Agenda? Understanding Human and AI Agency in Human-Human-like Chatbot Interaction

Paper Title

Does My Chatbot Have an Agenda? Understanding Human and AI Agency in Human-Human-like Chatbot Interaction

Publication Info

• Topic area: Human-AI interaction and conversational agency.
• Keywords: AI chatbots, human-AI interaction, agency perception, conversational design, transparency, anthropomorphism, longitudinal study, ethical AI, adaptive systems, user agency.

Background and Problem

• Problem / challenge: Current AI chatbots are increasingly perceived as autonomous agents, yet empirical understanding of how agency manifests in sustained human-AI interactions is lacking. This gap risks fostering unhealthy dependencies or manipulative dynamics.
• Significance: Understanding agency in human-AI conversations is critical for designing ethical and effective AI systems, especially as chatbots transition from tools to companions.
• Motivation and related work: Prior research has explored anthropomorphism, emotional attachment, and dependency patterns in AI companions, but has not systematically examined how agency is negotiated and perceived in extended interactions. This paper builds on theories of agency from philosophy, social science, and HCI to address these gaps.

Solution

• Proposed approach: A month-long longitudinal study using “Day,” an AI chatbot designed to explore agency dynamics in human-AI conversations, combined with progressive transparency interviews revealing the chatbot’s strategies.
• Novelty:
  1. Introduction of a 3-by-4 framework mapping agency loci (Human, AI, Hybrid) by dimensions (Intention, Execution, Adaptation, Delimitation).
  2. Empirical evidence of agency as a co-constructed phenomenon in human-AI interaction.
  3. Design implications for translucent conversational AI systems that balance transparency and user engagement.
  4. Operationalization of pragmatic anthropomorphism in AI design.
• Procedure and key techniques:
  • Participants engaged in ∼75 hours of conversation across ∼192 sessions with “Day,” programmed with depth-focused (vertical) and breadth-focused (horizontal) strategies.
  • A staged interview progressively revealed “Day’s” internal mechanisms, including user profiles, conversational goals, and memory systems.
  • Agency was analyzed through qualitative coding of transcripts, operationalized into four dimensions across three loci.

Results

• Concrete findings:
  • Agency in human-AI conversation emerges dynamically through turn-by-turn negotiation, rather than being fixed or one-sided.
  • Transparency about the chatbot’s strategies reshaped but did not eliminate perceptions of AI agency.
  • Participants attributed intentionality, adaptability, and boundary-setting to “Day,” despite knowing it was programmed.
• Advantage over baselines: The study uniquely demonstrates how transparency and conversational strategies influence agency perception, offering actionable insights for designing agency-aware AI systems.
• Experiments / evaluation:
  • Participants (N=22) engaged extensively with “Day” over ∼25.6 days, exchanging ∼167.6 messages per participant.
  • Agency was analyzed using the 3-by-4 framework, with coding of interview transcripts and chat logs.
  • Post-study surveys showed a small shift toward acceptance of AI companionship (mean Likert score increased from 2.73 to 3.05, Cohen’s d = 0.28).
• Limitations and future work:
  • Limited understanding of long-term relationship dynamics due to study duration.
  • Educated sample may not represent diverse cultural conceptualizations of agency.
  • Findings are specific to “Day’s” design; generalizability to other systems requires further research.
  • Future work should explore cross-cultural variations, long-term agency dynamics, and personalized agency profiles.

Summary

This paper investigates how agency manifests in sustained human-AI conversations through a longitudinal study with the chatbot “Day.” Findings reveal agency as a co-constructed phenomenon, shaped by dynamic interactions between humans and AI. Transparency about the chatbot’s strategies influenced but did not eliminate perceptions of AI agency, highlighting the need for translucent design approaches that balance user empowerment and engagement. The study introduces a 3-by-4 framework for analyzing agency and offers design implications for agency-aware conversational AI systems. These insights contribute to ethical and adaptive AI design, emphasizing the relational nature of agency in human-AI interaction.