Designing Scaffolding Strategies for Conversational Agents in Dialog Task of Neurocognitive Disorders Screening

Title of the Paper

Scaffolding Strategies for Conversational Agents in Dialogue Tasks for Neurocognitive Disorder Screening

Bibliographic Information

• Subject Areas: Human-Computer Interaction, Health and Aging, Applications of Artificial Intelligence in Healthcare
• Keywords: Health, Aging, Scaffolding, Conversational Agents, Neurocognitive Disorder Screening

Research Background and Problem

• Identified Problems or Challenges:
  • Neurocognitive disorders (NCDs), such as Alzheimer's disease, are prevalent among older adults, imposing a significant burden on patients, families, and healthcare systems.
  • Existing human-administered screening methods face scalability challenges, such as limited clinical resources and patients' mobility constraints.
  • Current conversational agent (CA) screening systems often employ simple dialogue strategies, but communication with cognitively impaired users requires more sophisticated techniques, such as simplified language, repetition, and scaffolding.
• Why This Problem is Important:
  • Early diagnosis and intervention can effectively manage NCD-related symptoms, alleviating societal and healthcare pressures.
  • Developing scalable screening methods can support routine cognitive monitoring and long-term tracking of cognitive functions.
• Research Motivation and Related Work:
  • Scaffolding theory has been shown to be crucial in facilitating activity comprehension and engagement for NCD patients, but its design and application in dialogue tasks remain underexplored.
  • Enhancing CA capabilities by mimicking clinical professionals' scaffolding practices can help build a dialogue system tailored to NCD patients.

Solution

• Proposed Method or Solution:
  • Based on scaffolding theory and clinical professionals' practices, analyze scaffolding strategies from video recordings of human-administered tests and propose a scaffolding framework for CAs.
  • Design a CA using a three-level scaffolding framework, including activity-level, action-level, and repair-level scaffolding.
  • Develop a semi-automated CA system that leverages ChatGPT (gpt-3.5-turbo) to support the retrieval and reasoning of core scaffolding strategies.
• Innovative Contributions:
  • The first study to explore how CAs can support NCD screening tasks through scaffolding strategies.
  • Combines scaffolding theory with users' cognitive development states (Zone of Proximal Development theory) to dynamically adjust scaffolding strategies based on users' task comprehension levels.
  • Proposes a framework and specific design processes to improve CA performance in cognitive impairment screening tasks.
• Implementation Steps and Key Technologies:
  • Conduct content analysis of dialogues between clinical professionals and participants to extract human scaffolding strategies.
  • Use an iterative design process to develop a CA prototype and evaluate the applicability of its scaffolding framework.
  • Integrate ChatGPT into the system to support dynamic question generation and optimization of scaffolding strategies.

Research Outcomes

• Specific Results:
  • The proposed CA was able to provide dynamic scaffolding, which was rated as appropriate in 89.45% of cases by professional clinicians.
  • Experiments showed that the use of scaffolding by the CA decreased progressively as tasks advanced, indicating that the scaffolding strategies facilitated users' task comprehension.
  • 68.25% of the scaffolding strategies recommended by ChatGPT were rated as appropriate by clinical experts.
• Advantages Over Existing Solutions:
  • Compared to existing NCD screening methods (e.g., tablet- or touchscreen-based), the proposed method better encompasses language and cognitive function testing.
  • The dynamic adjustment of the scaffolding framework allows the CA to tailor scaffolding strategies based on users' cognitive states, improving task completion rates.
• Experimental or Evaluation Results:
  • User studies involved 15 participants (including healthy individuals, and those with mild and major cognitive impairments), with each participant completing at least one valid task (average of 2.87 tasks completed).
  • Most participants found the CA's scaffolding helpful for task comprehension and easy to use.
• Limitations and Future Directions:
  • The sample size was small, and future studies should expand the sample size and conduct statistical validation.
  • The current CA is semi-automated and requires further development into a fully automated version to validate the reliability of screening results.
  • The study focused on a specific task (GSDT); future research should evaluate its generalizability to other cognitive screening tasks.
  • Although the system's manual operation delays were deemed acceptable, further optimization is needed to support smooth user navigation during tasks.

Conclusion and Design Implications

• Integrate a three-level scaffolding framework (activity-level, action-level, repair-level) into CA design to meet users' needs at different stages.
• Dynamically adjust the frequency and type of scaffolding based on users' Zone of Proximal Development states.
• For future automated CA development, leverage visual and language models to better handle language and visual information in complex scenarios.