MagFace: Interference-Resistant Facial Gesture Recognition System on Cycling Glasses with Low-Power Magnetic Sensing

Paper Title

MagFace: Interference-Resistant Facial Gesture Recognition System on Cycling Glasses with Low-Power Magnetic Sensing

Publication Info

• Topic area: Wearable technology for hands-free interaction in cycling environments
• Keywords: Facial gesture recognition, magnetic sensing, cycling glasses, wearable interaction, deep learning, low-power sensing, interference resistance, user-defined gestures, real-time system, urban cycling

Background and Problem

• Problem / challenge: Existing wearable facial gesture recognition systems struggle with interference from lighting, vibration, sweat, noise, and temperature changes in outdoor cycling scenarios. Additionally, prior gesture sets are often complex and unsuitable for cycling.
• Significance: Safe and efficient hands-free interaction is critical for cyclists to avoid distractions and maintain control, especially in dynamic environments.
• Motivation and related work: Previous methods using optical, capacitive, physiological, or acoustic sensing are sensitive to environmental factors such as sweat, motion, and lighting. Magnetic sensing offers robustness against these interferences but has not been fully explored for facial gesture recognition in cycling contexts.

Solution

• Proposed approach: MagFace, a wearable facial gesture recognition system integrated into cycling glasses, using magnetic silicone inserts and magnetometers combined with a deep learning pipeline for robust gesture classification.
• Novelty:
  1. Integration of magnetic sensing into cycling glasses for interference-resistant facial gesture recognition.
  2. Development of a tailored deep learning pipeline with anti-noise preprocessing and voting mechanisms for real-time interaction.
  3. Design of a user-defined facial gesture set optimized for cycling scenarios.
• Procedure and key techniques:
  • Magnetic silicone inserts placed on glasses frames capture facial muscle movements.
  • Preprocessing includes magnetometer calibration, thermal compensation, filtering, and normalization.
  • A lightweight 1D-CNN pipeline classifies gestures in two stages: binary detection and six-class recognition.
  • Voting mechanisms reduce false positives and ensure reliable real-time interaction.

Results

• Concrete findings:
  • Stationary scenario: F1-score of 0.971 (UD) and 0.898 (UI).
  • Cycling scenario: F1-score of 0.973 (UD) and 0.904 (UI).
  • Cross-scenario evaluation: F1-score of 0.976 (UD) and 0.922 (UI).
  • Controlled conditions: Accuracy of 0.841–0.897 under strong lighting, wind, bumpy roads, and uphill cycling.
  • Real-time urban cycling: Accuracy of 85.9%, false positive rate of 4.07 per hour, latency of 643.11 ms.
• Advantage over baselines: Compared to optical, capacitive, and acoustic sensing methods, MagFace demonstrated superior robustness to environmental interferences and lower power consumption (150 mW peak).
• Experiments / evaluation:
  • Gesture elicitation study (N=22) defined a user-centered facial gesture set.
  • Performance evaluation across stationary (N=15), cycling (N=15), and controlled conditions (N=8).
  • Real-time usability study in urban cycling (N=14) compared MagFace to voice interaction (Siri).
• Limitations and future work:
  • Challenges in real-world deployment due to behavioral adaptations and environmental constraints.
  • Limited compatibility with prescription glasses.
  • Future plans include adaptive voting mechanisms, higher-density sensor arrays, and integration into other wearable devices like MR headsets and diving goggles.

Summary

MagFace is a wearable system for interference-resistant facial gesture recognition, designed specifically for cycling glasses. It combines magnetic sensing with a deep learning pipeline to achieve high accuracy and robustness against environmental challenges such as lighting, wind, and vibration. Evaluations demonstrated strong performance across stationary, cycling, and real-world urban scenarios, achieving an average accuracy of 85.9% in real-time use. The system offers a safe, natural, and hands-free interaction modality for cyclists, with potential applications in delivery work, underwater interaction, laboratory settings, and mixed reality environments. Future work will address sensor density optimization, adaptive voting mechanisms, and broader integration into wearable devices.