Designing a Personalised Sensor Glove Using Deep-Learning

Document Title

Designing a Sensor Glove Using Deep Learning

Document Information

• Subject Area: Application of deep learning in wearable device design
• Keywords: Sensor selection, wearable design, computational design, gesture detection, deep feature selection, deep learning

Research Background and Issues

• Problems or Challenges:

  1. In smart glove devices, the type and layout of sensors are crucial for gesture recognition accuracy, but existing studies fail to comprehensively identify the most critical sensors.
  2. There is a need to explore how to remove certain sensors without significantly reducing performance under cost-reduction requirements.
  3. Individual differences in user gestures require personalized device design, but such personalized optimization has not been fully explored.

• Importance: Gesture recognition has broad applications in fields such as graphic animation, rehabilitation therapy, game control, and virtual/augmented reality interaction. Optimizing sensor glove design not only improves gesture recognition accuracy but also reduces hardware redundancy and costs.

• Research Motivation and Related Work:

  • Although there are existing studies using traditional machine learning or deep learning for gesture recognition, there is currently no clear method for sensor selection and optimization.
  • Current methods fail to provide reliable design decisions, such as identifying which sensors contribute most to the task.
  • The authors reference feature selection and network pruning methods as prior work but point out the limitations in model interpretability.

Solution

• Proposed Method:

  • Developed a deep learning-based sensor importance estimation method to evaluate the contribution of sensors to gesture recognition through feature selection.
  • Added an extra sparse layer (Importance Estimation Layer, IE layer) to the deep learning model to analyze and rank sensor importance.
  • Extended the method by introducing a second layer (Cost Importance Estimation Layer, CIE layer) to incorporate sensor cost into the optimization objective.

• Innovations:

  • Introduced the Importance Estimation Layer (IE) to rank and analyze sensor data channels, enabling designers to understand sensor contributions to classification tasks.
  • Proposed a novel multi-objective optimization approach (accuracy and cost), combining hardware cost with task accuracy.
  • Achieved both general and personalized optimization, supporting the design of gesture gloves for different users.

• Implementation Steps and Key Techniques:

  1. Data Collection: Captured motion data using an experimental glove equipped with 28 sensors.
  2. Model Training: Built a deep learning model based on LSTM, embedding the Importance Estimation Layer between the input and the first layer.
  3. Feature Selection: Ranked sensor importance by analyzing IE layer weights and verified which sensors could be removed.
  4. Cost Optimization: Embedded sensor cost weights into the loss function through the CIE layer, enabling joint optimization of prediction accuracy and hardware cost.
  5. Validation: Tested the model performance by reducing certain sensors, confirming the effectiveness of the importance ranking and selection strategy.

Research Results

• Specific Results:

  • Effectiveness Validation: Tested sensor reduction in the range of 10%-60%, finding that gesture recognition accuracy remained within baseline levels when up to 40% of sensors were removed.
  • Personalization vs Generalization: Different users had varying requirements for sensor importance, with personalized optimization significantly improving prediction accuracy in some cases.
  • Cost Optimization: By introducing cost constraints, the optimized design was more cost-efficient in terms of hardware consumption.

• Advantages:

  • Provides a systematic, data-driven method for designers to make informed decisions about sensor layout and quantity based on weight analysis results.
  • Supports hardware cost optimization, allowing flexible design adjustments based on budget.
  • Enhances model interpretability within deep learning frameworks, making design decisions more transparent.

• Limitations and Future Directions:

  1. The method requires hardware design and data collection based on all sensors, resulting in high initial configuration costs.
  2. Personalized designs may not outperform generalized designs for certain users, requiring further investigation.
  3. The behavior of IE layer weights (e.g., the meaning of negative weights) has not been fully studied.
  4. Future research could explore the possibility of generating virtual gesture data to support more experimental scenarios.

Application Prospects:

• The method can be extended to other wearable devices (e.g., smart clothing) to optimize sensor layouts and reduce production costs.
• It can be applied to sensor configuration analysis for complex activity recognition tasks, such as evaluating the contribution of multiple IMUs in human activity monitoring.

Through this innovative approach, the paper provides new directions for optimizing wearable device design, particularly in sensor optimization and deep learning interpretability. Its methodology has the potential to further advance the field.