VibWalk: Mapping Lower-limb Haptic Experiences of Everyday Walking

Research Background and Problem Statement

• Identified Problems or Challenges: Existing research primarily focuses on foot pressure sensing and lower limb interaction, but the exploration of tactile information decoding through foot vibration remains limited, especially regarding the broadband vibration signals generated by foot-ground contact during walking and their decoding.
• Significance of the Problem:
  • The human plantar surface is a sensitive skin area capable of perceiving dynamic skin deformation and contact information during walking. This tactile information is crucial for walking safety, particularly in managing fall risks among the elderly.
  • Plantar tactile perception can assist in identifying ground materials and conditions, with potential applications in urban road monitoring, pedestrian navigation, and robotic learning.
• Research Motivation and Related Work:
  • Although wearable devices for applying pressure to the foot or tracking gait already exist, research on vibration sensing and tactile perception is still in its early stages.
  • The use of vibration signals for ground classification or tactile feedback in augmented reality has not been fully explored.

Proposed Solution

• Proposed Method:

  1. A wearable foot system named VibWalk is proposed to capture broadband vibration signals during walking activities.
  2. Integration of two types of sensors, an accelerometer (ACC) and a microphone (MIC), to record low-frequency (0–800Hz) and high-frequency (35–18000Hz) vibration signals, respectively.
  3. Development of a data-driven deep learning model (based on the ResNet framework) to decode tactile information.
  4. Integration with GPS positioning to achieve spatial mapping of tactile information.

• Innovative Aspects:

  • Fusion of broadband vibration sensing (ACC and MIC) to capture rich vibration features.
  • Introduction of a deep learning model combining time-domain and spectral-domain features, significantly improving ground material recognition accuracy.
  • Development of a tactile map generation mechanism to annotate ground materials, surface textures, and gait information.

• Implementation Steps:

  1. Hardware design: Combines custom 3D-printed plates, dual accelerometers, a microphone, an RGBD camera, and a Raspberry Pi.
  2. Data collection: Data from 31 participants walking on 18 types of surface materials, recording a total of 31 hours of vibration information.
  3. Feature extraction: Spectral features of ACC and MIC are extracted using STFT, integrating time-domain and spectral features.
  4. Model training: An improved ResNet model is employed to classify 18 types of ground materials.
  5. Tactile information visualization: Tactile maps are generated using GPS data, supporting multi-client collaborative data collection.

Research Outcomes

• Specific Results:

  1. Classification accuracy for earth materials reached 95.4% (within-user) and 87.1% (cross-user).
  2. Accuracy for distinguishing between wet and dry surfaces was 96.9% (MIC) and 90.1% (ACC).
  3. Vibration spectral features effectively distinguished particle sizes (R² > 0.93), supporting road damage detection.
  4. Real-time spatial mapping of tactile information was achieved, enabling the generation of tactile maps annotated with ground material categories.

• Advantages:

  • Outperforms traditional solutions using a single accelerometer or microphone.
  • Simultaneously extracts gait and ground vibration features, providing comprehensive tactile sensing capabilities.
  • Scalable for applications in urban management (e.g., detecting road puddles or cracks) and immersive tactile feedback in simulated environments.

• Experimental or Evaluation Results:

  • Classification accuracy using a single MIC was 92.9% (within-user) and 85.9% (cross-user), while ACC achieved 81.2% (within-user) and 70.6% (cross-user).
  • The fusion of MIC and ACC improved performance: 96.1% (within-user) and 88.5% (cross-user).

• Limitations and Future Directions:

  1. The model exhibits confusion in identifying certain materials (e.g., asphalt, concrete, and slate), requiring improvements in classifier generalization.
  2. Microphone sensors are significantly affected by environmental noise, necessitating enhanced signal robustness.
  3. The current device has limitations in user convenience (e.g., requiring a fixed vibration transmission plate). Future plans include miniaturization and designs better suited for daily wear.
  4. Expanding the scope of research to include more ground materials and human activity recognition, supporting distributed data processing through edge computing.
  5. Exploring tactile feedback devices to provide immersive experiences for users in virtual and augmented reality environments.

Conclusion

VibWalk effectively addresses key challenges in plantar tactile research by integrating noise-reducing sensing hardware, efficient data-driven models, and spatial mapping technologies. The research outcomes not only expand the application boundaries of human-computer interaction and intelligent devices but also provide valuable references for the design of future tactile feedback systems.