"Alone and Adrift in Analytics" - Insights from Long-term Involvements with stroke Clinicians when Using Care Quality Monitoring Systems

Paper Title

'Alone and Adrift in Analytics' - Insights from Long-term Involvements with Stroke Clinicians when Using Care Quality Monitoring Systems

Publication Info

• Topic area: Care quality improvement systems (CQIS) and their use in healthcare analytics.
• Keywords: Care quality improvement, CQIS, data visualization, healthcare analytics, stroke care, RES-Q, human-computer interaction, automation, clinical decision-making, exploratory data analysis.

Background and Problem

• Problem / challenge: Current CQIS face challenges such as overwhelming data, limited analytical expertise among clinicians, fragmented communication, and lack of actionable insights. These issues hinder the effective use of CQIS and lead to stagnating care quality.
• Significance: Improving CQIS usability and functionality is critical for enhancing healthcare outcomes, reducing clinician burnout, and fostering data-driven decision-making in hospitals.
• Motivation and related work: Prior research has focused on individual aspects of CQIS, such as data visualization or decision-making, but lacks a multidisciplinary understanding of user needs and analytical challenges. This study aims to address these gaps by investigating the specific tasks, struggles, and successful practices of clinicians using a widely adopted CQIS, RES-Q.

Solution

• Proposed approach: A five-year qualitative investigation into the use of RES-Q, an international stroke care CQIS, to identify user needs, challenges, and design implications for improving CQIS.
• Novelty:
  1. A user-centred design investigation identifying personas, workflows, and communication challenges in CQIS use.
  2. A detailed task and workflow analysis using Munzner’s visualization task framework to pinpoint where current dashboards fail.
  3. Design implications for integrating data analysis interfaces and automating tasks to address specific user needs.
• Procedure and key techniques:
  • Conducted ten qualitative studies (five interviews and five workshops) with 74 neurologists across 34 countries.
  • Analyzed workflows, tasks, and user personas using thematic analysis and Munzner’s framework.
  • Iteratively refined insights with feedback from clinician co-authors.

Results

• Concrete findings:
  • Clinicians primarily use confirmatory analysis to identify underperforming indicators and exploratory analysis to investigate trends and outliers.
  • Common challenges include synthesizing fragmented insights, performing subgroup analyses, and addressing follow-up questions during discussions.
  • Automation and conversational AI were identified as potential solutions to reduce repetitive tasks and facilitate quick explorations.
• Advantage over baselines:
  • RES-Q’s updated interface (e.g., filtering options, combined insights) moderately improved workflows compared to its earlier version.
  • Suggested improvements (e.g., automated alerts, customizable workflows, and conversational AI) aim to address broader usability and analytical challenges.
• Experiments / evaluation:
  • Activities included interviews, workshops, and prototype evaluations to gather user feedback on RES-Q’s interface and proposed solutions.
  • Participants spanned diverse roles and regions, ensuring a comprehensive understanding of CQIS use.
• Limitations and future work:
  • Limited generalizability due to the study’s focus on RES-Q and a predominantly European participant base.
  • Future research should explore the impact of exploratory analysis interfaces, automation, and conversational AI in CQIS across other domains and regions.

Summary

This study provides a comprehensive investigation into the use of RES-Q, a stroke care CQIS, identifying key challenges such as overwhelming data, fragmented insights, and limited analytical expertise. By analyzing user personas, workflows, and tasks, the study highlights the need for automation, customizable workflows, and conversational AI to support clinicians in care quality monitoring. The findings offer actionable design implications for improving CQIS usability and fostering data-driven healthcare improvements. These insights are applicable to other healthcare domains and can inform the development of adaptive and collaborative CQIS systems.