A Human-AI Collaborative Approach for Clinical Decision Making on Rehabilitation Assessment

Title of the Paper

A Human-AI Collaborative Approach for Clinical Decision Making on Rehabilitation Assessment

Paper Information

• Subject Area: Artificial Intelligence and Healthcare, Expert Decision Support Systems, Rehabilitation Assessment
• Keywords: Human-AI Interaction/Collaboration, Decision Support Systems, Explainable and Interactive Machine Learning, Personalization, Stroke Rehabilitation Assessment

Research Background and Issues

• Identified Problems or Challenges:

  • Current rehabilitation therapists often rely on clinical tests to assess patients' recovery status, which involves direct and visual observation. This process is time-consuming and infrequent.
  • Due to limited time, therapists lack quantitative data on patients' performance and recovery progress, making it difficult to make well-informed decisions.
  • Existing sensor-based and machine learning algorithm-driven rehabilitation monitoring systems, while feasible in laboratory settings, face adoption challenges due to non-user-centric design and the "black-box" nature of algorithms.

• Importance of the Problem:

  • Improving the quality of rehabilitation assessments is crucial for designing appropriate interventions and enhancing patient recovery outcomes.
  • Combining artificial intelligence with expert collaboration can provide more quantitative insights for clinical decision-making, thereby improving therapists' assessment consistency.

• Research Motivation and Related Work:

  • Previous studies have shown that transparency, explainability, and user involvement in human-AI collaboration are critical factors for improving the adoption of clinical decision support systems.
  • While human-AI collaboration has demonstrated potential in other domains, research in the field of rehabilitation assessment remains limited.

Solution

• Proposed Method or Solution:

  • The authors propose an interactive AI system that combines machine learning models with therapist-provided rule-based models for stroke rehabilitation assessment.
  • The system can automatically select significant movement features for assessment and present patient-specific analyses (e.g., "movement quality" predictions and feature comparisons) through a visualization interface.
  • Therapist feedback is used to further optimize the model, supporting personalized assessments.

• Innovative Contributions:

  • Integration of machine learning models and rule-based models into a hybrid model to enhance predictive capabilities and system transparency.
  • Dynamic feature selection implemented via reinforcement learning, enabling optimization of feature selection based on personalized patient data.
  • Development of an interactive patient-specific analysis interface to improve therapists' understanding and facilitate feedback for system refinement.

• Implementation Steps and Key Technologies:

  1. Dynamic Feature Selection:
     • Significant movement features are selected using reinforcement learning's Markov decision process.
     • Implementation includes Q-network, experience replay, and soft updates to target networks.
  2. Machine Learning Models:
     • Evaluation models are trained using various machine learning algorithms (e.g., decision trees, linear regression, support vector machines, and neural networks).
  3. Rule-Based Models:
     • Semi-structured interviews with therapists are conducted to identify and extract if-then rules related to rehabilitation assessment.
     • The model is updated in real-time to enable personalized assessments.
  4. Hybrid Model:
     • Machine learning models and rule-based models are combined using weighted averaging.
  5. Visualization Interface Design:
     • Provides patient movement videos, visual features, and detailed trajectory trend comparisons.
     • Supports therapists in providing feature-based feedback.

Research Outcomes

• Specific Results:

  • Compared to traditional systems, the proposed system significantly improved therapists' consistency in assessment tasks (traditional system average F1 score: 0.66; new system: 0.71, p < 0.05).
  • With therapist feedback, the system's performance further improved (initial average F1 score: 0.8377; increased to 0.9116, p < 0.01).

• Advantages:

  • Enhanced consistency and efficiency in therapists' assessments.
  • System explainability makes it easier for therapists to accept and refine the system for personalized decision-making.
  • The flexibility of the hybrid model allows performance optimization through human-AI collaboration.

• Experimental or Evaluation Results:

  1. User evaluations indicated that therapists perceived a significant reduction in workload (effort subscore decreased from 3.93 in traditional systems to 2.66, p < 0.10).
  2. In interactive experiments, therapists provided an average of 9 feedback items to optimize the system, with feature additions accounting for the highest proportion (7.26).
  3. The optimized hybrid model (HM10) outperformed standalone machine learning models (ML-NN) and the consistency level among therapists.

• Limitations and Future Directions:

  • Limitations:
    • This study focuses on a single domain (stroke rehabilitation assessment) without horizontal validation across other clinical scenarios.
    • Lack of long-term system deployment data makes it difficult to evaluate system stability in real-world interventions.
  • Future Directions:
    • Explore the application of this interactive AI system in other medical decision support domains.
    • Design more universal feedback mechanisms to support collaboration between different types of experts and systems.
    • Conduct long-term, cross-domain testing to assess the system's impact on clinical practice.

This paper provides novel insights into improving human-AI collaboration in the healthcare domain, particularly by enhancing the practicality of AI in rehabilitation assessment through explainability and interactive design.