RadarFoot: Fine-grain Ground Surface Context Awareness for Smart Shoes

Title of the Paper

RadarFoot: Fine-grain Ground Surface Context Awareness for Smart Shoes

Paper Information

• Field of Study: Ground surface recognition in wearable technology and smart shoe applications
• Keywords: Wearable devices, smart shoes, mmWave radar, context awareness, machine learning, ground surface recognition, wet/dry detection

Research Background and Problem

• Problems and Challenges:
  • Current smart shoes lack advanced ground surface context awareness, which is critical for user functionality and experience.
  • Existing technologies for ground surface recognition are limited to specific scenarios or conditions and cannot handle ecologically valid data or diverse surfaces in dynamic environments.
  • There is a need for a solution capable of dynamically detecting multiple surface types and user activities, such as gait changes.
• Research Motivation:
  • More accurate ground surface recognition can support applications such as navigation for the visually impaired, indoor positioning, and detection of wet or slippery surfaces.
  • The high reflectivity of radar makes it particularly suitable for detecting surface material properties, offering new possibilities for ground surface classification.

Solution

• Main Approach or Solution:
  • Propose a ground surface classification method using millimeter-wave radar (mmWave Radar) embedded in smart shoes.
  • Model surface characteristics based on radar reflection signals and design machine learning classifiers to identify surface types under both dynamic and static conditions.
• Innovations:
  • Ground surface classification in dynamic environments (e.g., "dynamic classification" while walking), integrating mmWave radar technology into footwear for the first time.
  • Development of a system for wet/dry surface recognition, addressing limitations of existing visual methods.
• Implementation Steps and Key Techniques:
  1. Hardware Development:
     • Utilize the Acconeer XR112 mmWave radar sensor, Raspberry Pi 4 control board, Pisugar battery, and an IMU to validate motion ground truth.
     • Adjust parameters (e.g., signal gain, update frequency) to optimize the quality of radar echo signals.
  2. Data Collection and Feature Extraction:
     • Collect "in-the-wild" data from 23 participants walking on campus, along with static data in a laboratory setting.
     • Define 14 features (e.g., echo energy at different depths, signal gradients) to describe the structure of reflected wave signals.
  3. Model Development:
     • Train two main classifiers—dynamic surface classification model and static surface classification model—using machine learning models such as Random Forest (RF) and SVM.
     • Introduce gait cycle detection to enable real-time feature extraction and automated classification.
  4. Wet/Dry Surface Detection:
     • Simulate wet surface conditions to enhance the model's recognition capabilities.
  5. Real-time Implementation:
     • Develop an efficient detection thread for real-time response to gait and surface classification, with an average latency of approximately 1 second.

Research Results

• Main Findings:
  1. Under dynamic conditions, the accuracy for 5 surface types is 80%, and for 10 surface types, it reaches 66.25%.
  2. Under static conditions, the accuracy for 10 surface types is 98.1%, while wet/dry surface classification accuracy is 95.6%.
  3. Activity type detection (e.g., standing, walking, climbing stairs) achieves an accuracy of 90.22%.
• Comparison with Existing Solutions:
  • More suitable for dynamic conditions compared to some static surface detection methods (e.g., mSense or CapSoles), though slightly lower accuracy for certain surface types (e.g., 10-class classification).
  • Does not rely on vision or ambient light, surpassing traditional camera- or IMU-based methods.
• Experiments and Evaluation:
  • Experiments involve complex setups with various real-world surfaces.
  • User participation validates differences across genders and gaits.
  • Balances model performance and practical feasibility, significantly reducing reliance on perspective through feature engineering.
• Limitations and Future Directions:
  • The dynamic model has limited ability to distinguish between similar surfaces (e.g., gravel and dry mud).
  • Imbalanced and small datasets restrict the performance of deep learning models.
  • Current hardware design is bulky; future work could focus on miniaturization and energy optimization.
  • Future research should explore the impact of natural environmental factors such as high temperatures and humidity on model robustness.

Conclusion

This study successfully integrates millimeter-wave radar into smart shoes, achieving efficient ground surface classification under both dynamic and static conditions. The findings have practical applications in areas such as indoor positioning, navigation for the visually impaired, and cushioning adjustment for running shoes. Future research aims to incorporate more advanced data processing methods and hardware designs to further advance smart shoes and wearable devices.