DiaryHelper: Exploring the Use of an Automatic Contextual Information Recording Agent for Elicitation Diary Study

Title of the Paper

DiaryHelper: Exploring the Use of an Automatic Contextual Information Recording Agent for Elicitation Diary Study

Paper Information

• Research Area: Human-Computer Interaction, Applications of Generative AI in Qualitative Research
• Keywords: Diary Study Method, Generative AI Technology, Contextual Information, Episodic Memory, User Engagement

Research Background and Problem Statement

• Identified Problems or Challenges:

  1. Traditional methods for guiding participants to record contextual information about events have limitations, such as diary entries being incomplete or vague, which affects participants' ability to recall details during subsequent interviews.
  2. Current improvement methods, such as using external devices (e.g., cameras, sensors) for recording, face feasibility issues in long-term diary studies due to equipment costs and participant burden.

• Significance: Contextual information is crucial in memory elicitation processes, as it can enhance the quality of qualitative research data and the depth of insights. However, capturing event details in a low-burden and efficient manner remains a challenge.

• Research Motivation and Related Work:

  1. Systems designed based on episodic memory theory can optimize the recording process and improve recall quality.
  2. Using generative AI technology to automatically capture and supplement contextual information, thereby reducing participant burden, is a promising direction for exploration.

Proposed Solution

• Proposed Solution: An automated contextual information recording agent, DiaryHelper, was designed using generative AI technology to assist participants in capturing contextual information such as time, location, emotions, people, and activities related to events.

• Innovations of the Solution:

  1. Integration of generative language models (GPT-3.5) with cloud services (Microsoft Azure, Google Cloud) to process multimodal data.
  2. Provision of a lightweight user interaction interface that infers and generates contextual information to assist participants in memory recall based on their diary entries.

• Implementation Steps and Key Technologies:

  1. System Design: Selection of five contextual dimensions based on episodic memory theory (time, location, emotions, people, activities).
  2. Multimodal Data Processing:
     • Use of cloud APIs to analyze information from images, videos, and audio.
  3. Contextual Information Generation: Utilization of large language models to predict potential contextual labels based on diary data and generate editable memos for participants.
  4. User Interaction Design: Integration with Slack interface to automatically generate editable contextual information tables after participants input diary entries.

Research Outcomes

• Specific Outcomes:

  1. DiaryHelper helps participants capture rich and accurate contextual information while reducing the recording burden.
  2. Increased participants' willingness to record and their rate of immediate entries, encouraging the use of diverse recording formats (e.g., images, audio).
  3. During recall, participants were able to recount recorded events in greater detail and provided more meaningful research topic insights.

• Advantages over Existing Solutions:

  1. DiaryHelper's automated contextual information recording significantly reduces participant burden, making it more cost-effective and scalable compared to other solutions requiring external devices.
  2. The system is highly interactive, enhancing participants' empathy and engagement through features like emotional tagging.

• Experimental or Evaluation Results:

  1. Experimental data showed that with DiaryHelper, participants' diary entries were significantly richer and more detailed compared to baseline systems (e.g., improved recall in location and emotion dimensions).
  2. DiaryHelper achieved high prediction accuracy (e.g., F1 score of 0.69 for emotion dimension), with particularly strong performance in identifying people (accuracy up to 90%).

• Limitations and Future Directions:

  1. The system's applicability to specific types of diary studies, such as those involving special populations or highly complex scenarios, requires further exploration.
  2. The narrow age range of participants (22–28 years) may introduce bias, necessitating evaluation across a broader demographic.
  3. The system's performance in long-term diary studies and the impact of generated contextual information on participants' independent reflection need further investigation.
  4. The system currently does not support the processing of physical object data; future work could involve designing richer multimodal recording methods.