"Having Lunch Now": Understanding How Users Engage with a Proactive Agent for Daily Planning and Self-Reflection

Paper Title

'Having Lunch Now': Understanding How Users Engage with a Proactive Agent for Daily Planning and Self-Reflection

Publication Info

• Topic area: Human-AI interaction in productivity and well-being contexts.
• Keywords: Conversational agents, proactive systems, behavior change, planning, self-reflection, LLMs, human-AI interaction, productivity, well-being, dialogue analysis.

Background and Problem

• Problem / challenge: While proactive conversational agents have shown promise in supporting productivity and well-being, there is limited understanding of how users interact with these agents, particularly in coaching roles, and what leads to conversational breakdowns.
• Significance: Understanding user-agent interaction dynamics is critical for designing systems that foster meaningful behavioral change and avoid disruptions in communication.
• Motivation and related work: Prior research has explored task-oriented and well-being-focused conversational agents but has largely focused on outcomes rather than the conversational processes. Gaps remain in understanding user behaviors, resistance, and cooperation with proactive, LLM-based agents.

Solution

• Proposed approach: A proactive coaching agent named PITCH, designed to facilitate daily planning and self-reflection through morning and evening check-ins.
• Novelty:
  1. Thematic and dialogue-act analysis of user-agent interactions to uncover behavioral patterns.
  2. Deployment of a proactive, LLM-based coaching agent for in-situ study.
  3. Design recommendations for adaptive, context-aware conversational agents.
• Procedure and key techniques:
  • Conducted a 14-day longitudinal study with 12 graduate students.
  • PITCH initiated twice-daily conversations for planning (morning) and reflection (evening).
  • Analyzed 336 conversations (3,181 turns) using thematic coding, dialogue-act analysis, and sentiment analysis.
  • Compared fixed-goal and rotating-goal versions of PITCH to study engagement and breakdowns.

Results

• Concrete findings:
  • Users externalized plans (21.5% of turns) and reported progress (18.4%), with higher planning in the morning (40.1%) and reflection in the evening (22.2%).
  • Positive sentiment was higher in evening conversations (30.3% increase).
  • Common behaviors included compliance (8.6%), self-reflection (11.2%), and providing context (12.3%).
• Advantage over baselines:
  • PITCH facilitated structured planning and reflective behaviors, supporting prospective memory and self-awareness.
  • Users treated the agent as a social partner, engaging in negotiation, clarification, and updates.
• Experiments / evaluation:
  • Conducted with 12 graduate students (6 male, 5 female, 1 non-binary; ages 22–35) over 14 days.
  • Evaluated using dialogue-act frequencies, sentiment analysis, and thematic patterns.
• Limitations and future work:
  • Small, homogeneous participant pool (graduate students in STEM).
  • Short study duration (14 days) limits understanding of long-term engagement.
  • Future work should explore diverse populations, longer deployments, and agents with richer memory and meta-awareness.

Summary

This study investigated how users engage with a proactive, LLM-based coaching agent, PITCH, for daily planning and self-reflection. Through a two-week deployment, users demonstrated cooperative behaviors such as planning, reporting, and reflecting, while also negotiating and correcting the agent’s assumptions. Despite fostering self-awareness and accountability, breakdowns occurred due to rigid conversational goals, premature turn-taking, and hallucinated capabilities. The findings highlight the need for adaptive, context-aware, and socially intelligent conversational agents. Future research should explore longer deployments and diverse user groups to refine agent design for productivity and well-being contexts.