An Activity Recognition System for Taking Medicine Using In-The-Wild Data to Promote Medication Adherence

Title of the Paper

A Medication Intake Activity Recognition System Using Large-Scale Uncontrolled Environment Data to Promote Medication Adherence Research

Bibliographic Information

• Subject Area: Machine learning applications in health management, particularly activity recognition based on wearable technology, focusing on promoting medication adherence
• Keywords: Human activity recognition, wearable technology, medication adherence, daily life activities, smartwatches

Research Background and Problem Statement

• Problems or Challenges Identified by the Authors:
  1. Globally, up to 50% of patients fail to take medications as prescribed, leading to reduced treatment efficacy and increased healthcare costs.
  2. Existing intervention methods rely on electronic medication packaging, smart home devices, or mobile applications, but these approaches lack personalization, depend on user input, and struggle to automatically monitor real-world behaviors.
  3. Medication intake actions constitute a very small proportion of daily life activities (imbalanced data), and existing methods to address data imbalance (e.g., oversampling) perform poorly in this domain.
• Why This Problem is Important: Low medication adherence results in higher hospitalization rates, mortality, and healthcare costs (annual healthcare expenses in the U.S. due to this issue range from $100 billion to $300 billion). Improving medication adherence can enhance patient health management and alleviate pressure on healthcare systems.
• Research Motivation and Related Work:
  1. Previous studies have explored using wearable devices to recognize daily activities, but specific research on medication intake behavior remains limited and is often confined to controlled laboratory settings rather than real-world environments.
  2. The authors aim to develop an efficient and practical activity recognition system capable of:
     • Accurately identifying medication intake behavior in uncontrolled real-world environments;
     • Utilizing general-purpose mobile devices (e.g., smartwatches and smartphones) as sensor sources.

Solution

• Method or Solution:
  1. Proposed an improved activity recognition model based on ensemble learning techniques (referred to as "Controlled Bagging").
  2. Modified the "Bagging" algorithm to independently sample minority classes (medication intake activities) to address class imbalance issues.
  3. Introduced the concept of "Portfolio Classification," which refines classification results using prediction probability distributions generated by initial classifiers.
• Innovations:
  1. Studied "Portfolio Classification" based on activity probability distributions to improve prediction accuracy.
  2. Conducted multi-user activity recognition for medication intake behavior in uncontrolled, full-day scenarios.
  3. Overcame the limitations of traditional Bagging or Random Forest methods in handling extreme data imbalance by using the improved Bagging approach.
• Implementation Steps and Key Techniques:
  1. Data Collection:
     • Collected daily accelerometer data via smartwatches, covering various activities such as brushing teeth, eating, and medication intake.
     • Built an "unconstrained" non-laboratory environment dataset from the real-day activities of 9 participants.
  2. Data Preprocessing:
     • Applied a sliding window method to segment sensor data, with a window size of 1 second and 75% overlap, capturing fine-grained activity features.
     • Extracted 106 highly relevant features (including acceleration energy, entropy, spectral features, etc.).
  3. Classifier Design:
     • Used "Controlled Bagging" to address class imbalance, controlling the proportion of stratified random sampling internally (skew parameter).
     • Introduced a portfolio classification strategy to calibrate results using prediction probabilities from baseline models.
     • Compared various secondary classification strategies (e.g., SVM, Manhattan distance) to evaluate performance improvements.

Research Outcomes

• Specific Results:
  1. Achieved an F1 score of 0.77 for medication intake activity recognition in user-adaptive models, demonstrating high precision and recall in real-world environments.
  2. Proposed a novel "Portfolio Classification" method that effectively utilizes prediction probability patterns to refine classifier outputs.
  3. Demonstrated that the "Controlled Bagging" algorithm significantly improves classification performance for highly imbalanced data, outperforming standard Bagging and Random Forest methods.
• Advantages Compared to Existing Solutions:
  1. Capable of recognizing medication intake behavior in real-world uncontrolled environments, rather than being limited to laboratory settings.
  2. Optimized activity recognition models for handling minority class imbalance issues (e.g., performance degradation in traditional Bagging or Random Forest).
• Experimental or Evaluation Results:
  1. Individual-specific models achieved an average F1 score of 0.735 for minority classes, which increased to 0.77 with user adaptation.
  2. Various experimental comparisons demonstrated that models using "Controlled Bagging" performed significantly better on test sets than traditional algorithms like Random Forest.
  3. Secondary classification incorporating multiple distance metrics (e.g., cosine similarity) further improved model performance.
• Limitations and Future Directions:
  1. The current dataset is collected over a single day, and some activities (particularly medication intake) are simulated. Future work should validate system stability and scalability using real long-term data.
  2. Sensor data relies on wearing smartwatches on both wrists, but in real-world applications, some users may only wear a single smartwatch.
  3. Future development should focus on lightweight hardware-software integration modules, such as data collection via screenless wristbands, to enhance wearability.
  4. Expand the range of recognized activities and explore applications in other health management scenarios, such as exercise analysis or dietary management.

Through this research approach, the authors open new possibilities for designing intelligent healthcare systems. The core technologies and results lay a foundation for the future development of personalized health management applications.