CardioAI: A Multimodal AI-based System to Support Symptom Monitoring and Risk Prediction of Cancer Treatment-Induced Cardiotoxicity

Research Background and Problem

• Identified Issues or Challenges:

  1. Cardiotoxicity caused by cancer treatments (e.g., chemotherapy) is a major side effect, potentially leading to severe outcomes such as heart failure and arrhythmias.
  2. Early symptoms are often mild or even absent, typically only detected in late clinical stages, resulting in irreversible damage.
  3. Symptoms frequently occur outside clinical settings, with a lack of continuous monitoring and timely diagnostic tools.
  4. Current patient self-reporting systems are inaccurate; patients may underreport or misreport due to limited health awareness or memory issues.
  5. Physicians face heavy workloads, and rushed decision-making processes may overlook critical information, impacting early detection.

• Why This Problem is Important: Cardiotoxicity is a leading cause of mortality and long-term complications in cancer patients. Early diagnosis and intervention are crucial for improving patient outcomes. However, existing monitoring and management tools are insufficient to support timely decision-making by physicians.

• Research Motivation and Related Work:

  • Remote patient monitoring (RPM) and AI-based clinical decision support systems (AI-CDSS) have shown potential for monitoring patient health in non-clinical settings.
  • Related studies have developed AI-based systems to predict cardiac dysfunction caused by cancer treatments, but these often rely on incomplete clinical data, lack flexible workflow integration, and offer limited interpretability.

Solution

• Proposed Method or Solution: The authors developed a multimodal AI system—CardioAI—which integrates wearable devices and a large language model (LLM)-based voice assistant to continuously collect patients' physiological signals and self-reported symptoms. The system uses an explainable AI model to generate cardiotoxicity risk scores and related insights.

• Innovative Aspects of the Solution:

  1. Multimodal Data Integration: Combines physiological data from wearable devices, unstructured voice data from patient self-reports, and electronic health records (EHRs).
  2. Comprehensive Risk Assessment: Dynamically generates cardiotoxicity risk scores using AI predictive models and provides interpretable features to identify risk factors.
  3. Human-AI Collaborative Design: The system design follows user-centered principles, incorporating physician input to better integrate into clinical workflows.
  4. Remote Monitoring Capability: Enables real-time symptom monitoring across geographic distances, reducing patient visits and physician workload.

• Implementation Steps and Key Technologies:

  1. Data Collection: Physiological signals (e.g., heart rate, respiratory rate, and blood oxygen levels) are monitored using the Garmin Vivosmart 5 device, while patient voice reports are recorded via Amazon Echo smart speakers.
  2. Backend Processing:
     • A Transformer-based model processes multi-time-series data and generates risk predictions within a survival analysis framework.
     • Shapley value methods provide model interpretability, helping physicians understand key risk factors.
     • An LLM (GPT-4 version) generates daily health summaries and personalized voice interaction explanations for patients.
  3. Frontend User Interface: An interactive dashboard is designed to modularly display patient information, daily summaries, physiological data, risk scores, and explanations.
  4. User Evaluation and Optimization: Participatory design collects physician feedback to continuously improve system functionality and user experience.

Research Outcomes

• Specific Achievements:

  1. Delivered a complete prototype of an AI-assisted decision support system that integrates continuous monitoring, risk prediction, data synthesis, and interpretability.
  2. Achieved parallel data comparison (integration of physiological data and voice symptom reports), improving physicians' symptom recognition capabilities.
  3. Provided dynamic, interpretable cardiotoxicity risk scores to assist physicians in making more accurate decisions.

• Advantages Over Existing Solutions:

  • Compared to current decision support systems based on electronic health records, it offers more comprehensive real-time monitoring data.
  • Better aligns with human-AI collaboration principles, addressing physician workflow needs and reducing cognitive load.
  • Enhances physician trust and understanding through the inclusion of explainable AI.

• Experimental or Evaluation Results:

  1. Heuristic evaluations by four physicians (two cardiologists and two oncologists) demonstrated strong performance in usability (average SUS score of 72.33±1.89) and workload optimization (NASA-TLX effort dimension score of only 1.13±0.70).
  2. Physicians acknowledged the system's proactive decision-support model, highlighting its potential to alleviate information overload and improve clinical response times.

• Limitations and Future Directions:

  1. Limited generalizability due to small sample size and participation from a single hospital; future work should include experts from more specialties and multi-institutional data.
  2. The system has not yet achieved seamless integration with existing EHR systems.
  3. The patient-side modules (e.g., voice assistant and wearable device user experience) have not been fully evaluated.
  4. Further validation of the AI model's accuracy and data security is needed to ensure privacy protection and enhance physician trust.
  5. Proposed expansion of such multimodal AI systems to other medical domains, such as diabetes or cardiovascular disease monitoring.

In summary, the CardioAI system improves the monitoring and prediction of cardiotoxicity induced by cancer treatments. Combining cutting-edge technology with user-centered design principles, it provides a significant pathway for advancing human-AI collaborative healthcare technologies.