Robust and Deployable Gesture Recognition for Smartwatches

Title of the Paper

Robust and Deployable Gesture Recognition for Smartwatches

Paper Information

• Topic Area: Gesture recognition on smartwatches, focusing on enhancing robustness and deployability
• Keywords: Gesture, sensing, wearable devices, mobile devices, deep learning

Research Background and Problem

• Problems and Challenges:

  • Gesture recognition on smartwatches faces numerous challenges, including resource constraints and dynamic user conditions.
  • Daily events (e.g., wearing and removing the watch) significantly reduce gesture recognition accuracy.
  • Large architectures of deep learning models are unsuitable for devices with limited computational power.

• Importance:

  • The success of gesture recognition largely depends on its flexibility and adaptability to diverse users and environments.
  • Achieving efficient gesture recognition on resource-constrained devices can provide critical technical support for the wearable device field.

• Research Motivation and Related Work:

  • Current technologies achieve high accuracy but lack sufficient robustness in the face of user, task, and environmental variations.
  • Modern deep learning models like GPT-3 cannot be deployed on devices with limited computational capabilities, such as smartwatches.
  • Previous studies suggest that collecting more representative training datasets and improving model structures can enhance recognition robustness.

Solution

• Methods and Solutions:

  1. Improved Data Collection Methods:
     • Extend obstacle course data collection methods to encourage users to generate diverse and realistic gestures in simulated real-world environments.
     • Cover various variables that may affect recognition, such as walking posture and watch tightness.
     • Collect everyday gesture data that may cause misclassification, such as drinking water or making phone calls.
  2. Model Design and Optimization:
     • Propose a convolutional network-based deep learning model capable of classifying raw sensor data and generated spectrograms.
     • Use global average pooling and a minimal model architecture to reduce parameters and improve efficiency on resource-constrained devices.
     • Integrate time-series and spectral data to enhance robustness by leveraging complementary information.

• Innovations:

  • Optimize dataset quality to enhance model robustness.
  • Propose a model with two orders of magnitude lower complexity than existing methods while maintaining over 98% classification accuracy.
  • Significantly reduce model size and computational resource requirements, enabling direct deployment on smartwatches.

• Implementation Steps and Key Techniques:

  • Data Processing and Labeling: Slice and label raw data collected from sensors.
  • Model Architecture Design: Design a lightweight network architecture based on convolutional units (ConvUnit).
  • Training and Evaluation: Test model robustness using various loss functions, including cross-entropy loss and adaptive robust loss.

Research Outcomes

• Experimental Results and Specific Outcomes:

  • Data collection through scenario simulation and diverse sample generation significantly improved training data quality.
  • The fusion model based on time-series and spectral data achieved over 98% accuracy across various experiments.
  • In user independence tests, the model achieved an average accuracy of 98.2% for new users; in scenario robustness tests, accuracy ranged from 96% to 99% for new situations.

• Comparison with Existing Solutions:

  • Compared to previous solutions requiring hardware modifications (e.g., high sampling rates), this method uses a standard 100Hz sampling rate while maintaining similar high accuracy.
  • The model's size and computational complexity are significantly reduced, making it more suitable for resource-constrained devices.

• Limitations and Future Work:

  • The current study focuses on binary classification tasks; further work is needed to extend to multi-class tasks.
  • Explore multi-sensor integration schemes to further enhance robustness.
  • Investigate the potential of model personalization and transfer learning to ensure broader applicability and higher accuracy.

Additional Information

The dataset used in this study has been open-sourced and is available at https://userinterfaces.aalto.fi/robustgestures for further research and technological development.