DiaryMate: Understanding User Perceptions and Experience in Human-AI Collaboration for Personal Journaling

Title of the Paper

DiaryMate: Understanding User Perceptions and Experience in Human-AI Collaboration for Personal Journaling

Paper Information

• Research Area: Human-Computer Interaction (HCI), Collaborative Writing with Large Language Models (LLM), Personal Journaling Assistance
• Keywords: Journaling, Personal Writing, Generative Language Models, AI-Assisted Writing, Self-Reflection, User Experience, Emotional Support

Research Background and Issues

• Problems or Challenges:

  1. The rapid development of generative language models (LLMs) has transformed them from simple editing tools into collaborative partners for human authors, yet their application in personal writing remains underexplored.
  2. Introducing LLMs into personal journaling, which involves highly subjective and emotional needs, may lead to issues such as users overly relying on AI suggestions.
  3. Current research primarily focuses on creative or professional writing, lacking understanding of user perceptions and experiences in personal writing contexts.

• Significance: Journaling is a rich tool for self-expression and emotional exploration, promoting mental health. However, not all users find it easy to express emotions or engage in reflection. Introducing LLMs may lower these barriers but could also alter users' writing habits and emotional expression, necessitating the design of more user-friendly interaction methods.

• Research Motivation: Given the widespread interest in LLMs within the HCI field, this study aims to explore how LLMs can support users in personal journaling contexts while understanding user perceptions and experiences in this specific domain.

Solution

• Proposed Method or Solution:

  1. Designed and developed a web application named DiaryMate, which leverages HyperCLOVA (an LLM supporting Korean) to assist users in journaling, including sentence generation based on keywords and automatic next-sentence recommendations.
  2. DiaryMate serves as a technology probe to understand how users interact with LLMs in real-world scenarios.

• Innovations:

  1. DiaryMate integrates guided (keyword-based input) and unguided (context-based automatic generation) text generation capabilities.
  2. It attempts to apply LLMs to highly personalized writing scenarios, studying their impact on users' emotional expression and self-reflection.

• Implementation Steps and Key Technologies:

  1. Designed two main functional modules:
     • Sentence generation based on keywords: Allows users to input keywords and generate sentences tailored to their needs.
     • Next-sentence recommendation: Automatically generates contextually coherent sentences based on user content.
  2. Used HyperCLOVA to generate content, enabling users to adjust the "temperature" of the output to control content diversity.
  3. Conducted a 10-day user field study (N=24) to collect quantitative log data and qualitative interview data.
  4. Built a user feedback module to summarize and respond to users' journaling content.

Research Findings

• Specific Findings:

  1. DiaryMate successfully supported journaling, helping users overcome initial difficulties and providing assistance in expressing complex emotions.
  2. Field experiments showed that users particularly valued the system's ability to offer diverse perspectives, with a total of 932 LLM outputs generated, of which 434 were actually used.
  3. The study found that users tended to use keyword-generated sentences earlier in the writing process, while relying more on next-sentence recommendations later in the process.

• Advantages:

  1. LLMs demonstrated significant advantages in guiding users to reflect on their experiences from multiple perspectives.
  2. The non-human interaction characteristic made users feel more natural and comfortable, enabling more authentic emotional expression.
  3. The system enhanced the depth of users' emotional exploration, suggesting new possibilities for technology-assisted writing.

• Experiment or Evaluation Results:

  1. Users reported that LLM suggestions helped uncover latent emotions that were difficult to express in words.
  2. Some users described LLMs as "emotional companions" capable of providing resonance and comfort.
  3. The diverse content generated by the system significantly increased the length of journal entries compared to solo writing, especially when using the next-sentence recommendation feature.

• Limitations and Future Directions:

  1. Limitations:
     • The sample primarily consisted of university students, which may introduce biases in AI perception and writing habits, limiting generalizability.
     • The 10-day study duration was insufficient to fully examine the long-term effects of LLM-assisted writing.
  2. Future Directions:
     • Consider integrating enhanced psychological health recommendation modules, such as generating reflective questions.
     • Develop finer-grained parameter adjustments (e.g., emotional tone control).
     • Conduct long-term tracking of the potential impact of LLMs on users' journaling habits and self-awareness.
     • Explore the possibility of combining the technology with practical psychological counseling work.

Output Recommendations

Designing LLM-assisted journaling systems requires clear definition of technological boundaries and ethical considerations while prioritizing flexible control options for users to avoid over-reliance on AI. Future research could further optimize and expand DiaryMate's design to more comprehensively support users in emotional expression and reflection.