Scaffolded Turns and Logical Conversations: Designing Humanized LLM-Powered Conversational Agents for Hospital Admission Interviews

Research Background and Issues

• Identified Problems or Challenges
  This paper focuses on key issues in hospital admission interviews, including limitations in nurses' time and manpower, which may lead to interruptions, misunderstandings, and incomplete records during data collection. Additionally, existing conversational agents (CAs) based on large language models (LLMs) lack flexibility and human-like communication skills, making it difficult to meet patient needs. Issues such as misunderstandings, insufficient emotional resonance, and the generation of erroneous information (hallucination) are prevalent.

• Significance
  Admission interviews are a critical component of healthcare services, helping to establish trust between patients and medical staff, reduce medical errors, and provide a caring environment. However, inefficiencies and shortcomings in this process can directly impact the quality of patient care.

• Research Motivation and Related Work
  This study aims to propose innovative solutions through participatory design methods, leveraging feedback and expertise from healthcare professionals to enhance the performance of LLM-based CAs in human-like communication skills and information collection efficiency. The research background covers recent advancements in conversational AI, automated medical questionnaires, and linguistic requirements in doctor-patient dialogues.

Solution

• Proposed Methods and Innovations
  The authors developed an innovative LLM-based conversational agent (CA) incorporating two core functionalities:

  1. Dynamic Topic Management: Utilizing graph-based conversation flows to manage topics and adjust question sequences, making interactions more logical and adaptive.
  2. Context-Aware Conversation Scaffolding: Supporting personalized questioning and patient guidance through few-shot prompt tuning.
     These methods address the shortcomings of existing CAs in flexibility and emotional resonance, making the agent more aligned with human conversational habits.

• Implementation Steps and Key Technologies

  1. Participatory Design: Collaborating with nurses and volunteers to design communication strategies.
  2. Modular System Implementation: Comprising two major modules:
     • Micro-level (Single-turn Conversation Optimization): Customizing responses through emotion and positioning modules and scaffolded question modules.
     • Macro-level (Multi-turn Conversation Management): Managing overall topic flow, logic checks, and cross-validation of information using graph structures.
  3. Few-shot Learning Techniques: Integrating real dialogue data from healthcare professionals to fine-tune model prompts and generate high-quality questions.

Research Outcomes

• Specific Results

  1. Technical Evaluation (Performance Assessment Experiments): Comparing system performance with human-generated dialogues and baseline models, demonstrating superior accuracy, consistency, and user experience.
  2. User Studies: A comparative experiment involving 44 users showed that the improved CAs significantly enhanced user experience and data collection quality compared to existing solutions.

• Comparative Advantages Over Existing Solutions

  1. Clearer and more logical question content, with outstanding performance in dynamic topic management and targeted questioning.
  2. Enhanced emotional support capabilities, fostering better emotional connections with users.
  3. Significant improvement in data collection quality, with better completeness and consistency of recorded information.

• Experimental or Evaluation Results
  The system's average scores on evaluation metrics (e.g., information accuracy, conversational logic) were significantly higher than those of baseline models, and even outperformed human-generated questions in certain metrics. Furthermore, experiments revealed that the emotional support and dynamic management modules led to more positive user interaction experiences and increased trust.

• Limitations and Future Directions

  1. Technical Limitations: The current speech recognition module has limitations in capturing user emotions and semantic ambiguities, while the text-to-speech module lacks emotional expression.
  2. Functional Expansion: The system currently focuses solely on information collection and does not provide deeper medical advice or diagnoses. Future work could enhance user interaction quality by integrating multimodal sensing (e.g., camera and audio analysis).
  3. Privacy Concerns: Data processing relies on cloud-based systems, and further optimization is needed to protect patient privacy, such as deploying local models or adopting privacy-preserving technologies.

Conclusion

The paper proposes a human-centered LLM-driven conversational agent designed to optimize the efficiency and patient experience of hospital admission interviews. By employing dynamic topic management and context-aware scaffolding techniques, the system significantly improves data quality and interaction experience, offering valuable insights into the integration of healthcare automation and patient care. However, challenges such as data privacy and broader system adaptability remain areas for future research exploration.