DiagLink: A Dual-User Diagnostic Assistance System by Synergizing Experts with LLMs and Knowledge Graphs

Paper Title

DiagLink: A Dual-User Diagnostic Assistance System by Synergizing Experts with LLMs and Knowledge Graphs

Publication Info

• Topic area: AI-assisted diagnostic systems integrating LLMs, KGs, and expert feedback.
• Keywords: diagnostic assistance, large language models, knowledge graphs, human-in-the-loop, medical AI, patient-physician collaboration, guided dialogue, evidence visualization, clinical decision support, continuous knowledge evolution.

Background and Problem

• Problem / challenge: Existing diagnostic systems lack dual-user interaction, dynamic knowledge integration, and end-to-end workflows that effectively combine patient and physician needs. They often rely on static knowledge graphs and fail to incorporate continuous expert feedback, limiting their real-world applicability.
• Significance: Addressing these gaps can improve diagnostic efficiency, reduce communication burdens, and enhance healthcare accessibility, especially in resource-limited settings.
• Motivation and related work: Prior work has explored clinical decision support systems (CDSSs), knowledge graph (KG)-based reasoning, and large language models (LLMs) for medical tasks. However, these approaches often focus on isolated subtasks, lack explainability, and fail to integrate evolving medical knowledge or expert validation into a unified framework.

Solution

• Proposed approach: DiagLink, a dual-user diagnostic assistance system that synergizes LLMs, KGs, and medical experts to support both patients and physicians through guided dialogues, collaborative reasoning, and continuous KG evolution.
• Novelty:
  1. A dual-user interactive system that supports both patients and physicians with role-adaptive interfaces.
  2. A closed-loop reasoning framework integrating LLMs, KGs, and expert feedback for continuous knowledge evolution.
  3. Unified presentation of multi-source evidence to enhance interpretability and reduce cognitive load.
  4. Visualization and interaction designs tailored to optimize diagnostic workflows and trust.
• Procedure and key techniques:
  1. Medical History Collection: Guided dialogues transform patient input into semi-structured records.
  2. Collaborative Diagnosis: LLMs generate initial diagnoses, KGs refine them through structured reasoning, and experts validate and update the KG.
  3. Evidence Presentation: Multi-source evidence (textual, graphical, and reasoning-based) is integrated into a unified interface for physicians.
  4. Expert-in-the-Loop: Experts iteratively review and refine KG content, ensuring continuous knowledge improvement.

Results

• Concrete findings:
  • DiagLink improved patient satisfaction across dimensions such as emotional support (+1.08), understanding (+0.75), and revisit intention (+1.42) on a 5-point Likert scale.
  • Physicians’ workload decreased significantly (NASA-TLX score reduced from 12.46 to 6.75), with diagnostic time reduced from 18.6 to 7.8 minutes.
  • Diagnostic accuracy improved, with Top-1 correct diagnoses increasing from 7 to 9 and Top-3 from 9 to 11 out of 12 cases.
  • Unsafe-suggestion rate was 0%, and no significant anchoring effects were observed.
• Advantage over baselines:
  • Outperformed Baseline A (text-based interface) in patient satisfaction, physician workload, and diagnostic accuracy.
  • Comparable to Baseline B (delayed diagnostic predictions) in safety while maintaining efficiency.
• Experiments / evaluation:
  • Conducted a controlled user study with 12 simulated patients and 12 physicians across 12 clinical scenarios.
  • Evaluated patient satisfaction, physician workload (NASA-TLX), diagnostic accuracy, and safety metrics.
  • Expert interviews and case studies validated usability and clinical relevance.
• Limitations and future work:
  • Limited to non-emergency, adult internal medicine cases; usability for low-literacy or assistive-technology users was not assessed.
  • Real-world deployment and long-term KG evolution effects remain unexplored.
  • Future plans include multimodal interfaces, broader clinical domains, and real-world evaluations.

Summary

DiagLink is a dual-user diagnostic system that integrates LLMs, KGs, and expert feedback into a collaborative framework, enhancing diagnostic workflows for both patients and physicians. It improves patient satisfaction, reduces physician workload, and moderately boosts diagnostic accuracy through guided dialogues, evidence-based reasoning, and continuous KG evolution. Evaluated through user studies and expert interviews, DiagLink demonstrates potential for improving healthcare accessibility and equity, with future work aimed at expanding its scope and real-world applicability.