Understanding the Impact of Long-Term Memory on Self-Disclosure with Large Language Model-Driven Chatbots for Public Health Intervention

Title of the Paper

Understanding the Impact of Long-Term Memory on Self-Disclosure with Large Language Model-Driven Chatbots for Public Health Intervention

Paper Information

• Subject Area: Application of LLM-driven conversational systems in public health monitoring
• Keywords: Chatbot, Large Language Model (LLM), Open-domain conversational systems, Long-term memory (LTM), Public health, Health check calls, Social isolation

Research Background and Issues

• Identified Problems or Challenges:
  1. Current LLM-driven chatbots can support health disclosure through open-ended conversations but often lack the ability to retain individual information across sessions.
  2. Public health monitoring requires users to disclose information continuously, but users are often hesitant to share sensitive health information.
  3. LLM-driven chatbots fail to personalize follow-ups based on prior session information, which affects user engagement and trust.
• Importance of the Issues:
  • Current public health monitoring methods rely on repeated data collection to understand population health dynamics, which is often time-consuming and labor-intensive. Chatbots with long-term memory capabilities (e.g., CareCall) can enhance user health disclosure by providing personalized experiences, thereby supporting public health goals.
• Research Motivation and Related Work:
  • Research Motivation: To develop chatbots that better engage users and facilitate long-term health disclosure, providing solutions for public health monitoring.
  • Related Work: Previous studies indicate that memory-enhanced chatbots capable of dynamically updating memory content have the potential to offer better support and feedback during user interactions. However, the real-world impact of these concepts lacks empirical understanding.

Solution

• Proposed Method or Solution:
  1. Developed CareCall, a voice-interactive chatbot powered by HyperCLOVA, equipped with long-term memory functionality.
  2. CareCall's LTM stores and dynamically updates user-related thematic information, such as health status, daily diet, sleep quality, outdoor activities, and pet-related details.
  3. Utilized LLM to provide memory-triggering functionality in subsequent conversations, supporting personalized health monitoring in a public health context.
• Innovations:
  • Applied LTM to LLM-driven chatbots, enhancing memory persistence and coherence through dynamic memory management strategies.
  • Conducted the first field-based comparative analysis of long-term user interactions with memory-enabled and non-memory-enabled chatbot versions.
• Implementation Steps and Key Technologies:
  1. Optimized the chatbot's conversational capabilities by training on large-scale dialogue datasets using data augmentation techniques.
  2. Integrated a memory management layer to store and dynamically update user summary information across five LTM themes (health, meals, sleep, visited locations, and pets).
  3. Compared deployments of chatbots with and without LTM functionality across different user groups, analyzing session logs and participant feedback.

Research Findings

• Specific Outcomes:
  • Users of the LTM-enabled CareCall version exhibited significantly increased health disclosure (detailed disclosure on health-related topics increased by 31%-76%).
  • Positive user responses to the chatbot (e.g., expressions of gratitude and reduced negative feedback) significantly improved (frequency of gratitude increased by 19%-69%).
  • Session duration with LTM-enabled chatbots significantly increased (from approximately 75 seconds to 88 seconds).
• Comparative Advantages Over Existing Solutions:
  1. Compared to chatbots without long-term memory, LTM-enabled chatbots achieved longer and higher-quality health-related disclosures.
  2. The LTM functionality successfully reduced users' perceived sense of indifference during chatbot interactions by providing personalized engagement.
  3. Demonstrated higher user engagement within the framework of public health monitoring.
• Experimental or Evaluation Results:
  • Quantitative Analysis: Used mixed-effects models to analyze the depth of disclosure and user responses across the five LTM themes.
  • Qualitative Research: Summarized the specific impacts of LTM triggers on health disclosure and user impressions through case conversation logs and user interviews.
• Limitations and Future Directions:
  • Limitations: The sample group predominantly consisted of female users, and the study was limited to the cultural context of South Korea. Privacy concerns among some users affected disclosure behavior.
  • Future Directions:
    1. Conduct cross-national comparative studies to address cultural sensitivity.
    2. Explore improved memory-trigger designs to better manage user privacy.
    3. Investigate diversified follow-up designs for patients with chronic illnesses.

Conclusion

This study systematically analyzed how LLM-driven chatbots with long-term memory capabilities influence user health disclosure and perceptions of the chatbot in the context of public health interventions. The findings demonstrate that LTM not only enhances disclosure quality but also improves user experience. However, a balance must be struck between public health monitoring objectives and user privacy sensitivities.