TeethTap: Recognizing Discrete Teeth Gestures using Motion and Acoustic Sensing on an Earpiece

Title of the Paper

TeethTap: Recognizing Discrete Teeth Gestures Using Motion and Acoustic Sensing on an Earpiece

Paper Information

• Subject Area: Non-invasive interaction technology using earpieces, recognizing user gestures through teeth movements
• Keywords: Teeth gestures, no visual input, no hand input, motion sensing, acoustic sensing, earpiece

Research Background and Problem

• Issues and Challenges:
  • Most current input technologies require manual operation, which is inconvenient when hands are occupied (e.g., holding objects).
  • Traditional interaction methods may not meet the needs of individuals with physical disabilities.
  • Existing systems often rely on complex or highly invasive devices (e.g., cameras or sensors mounted on the face), which complicates hardware design and may affect daily comfort.
  • Acoustic sensors are prone to noise interference, and motion sensors may fail during user movement.
• Research Significance:
  • Provides a lightweight, low-intrusion, flexible hands-free interaction method, expanding input possibilities for devices.
  • Advances research on cross-modal integration of sensing technologies.
• Research Motivation and Related Work:
  • Investigates the use of acoustic and motion sensors to solve the problem of teeth gesture recognition.
  • Combines multiple sensing technologies to improve gesture recognition accuracy and resistance to interference.

Solution

• Method/System Overview:
  • Proposes TeethTap, a wearable earpiece capable of capturing 13 discrete teeth gestures using motion sensors and acoustic sensors.
  • Employs a Support Vector Machine (SVM) to filter noise and combines it with a Dynamic Time Warping (DTW) algorithm for gesture recognition.
• Innovations:
  • Combines motion sensing (IMU sensors) and acoustic sensing (contact microphone) to mitigate the limitations of single technologies.
  • Designs a comprehensive gesture set that includes positions (e.g., front, left, right, back) and contact methods (single tap, double tap, hold).
  • Provides a customizable, easy-to-wear 3D-printed earpiece, effectively reducing device intrusiveness.
• Implementation Steps and Techniques:
  1. Hardware Design: Utilizes a 3D-printed earpiece equipped with IMU sensors and contact microphones behind both ears.
  2. Data Processing Pipeline:
     • Segments data windows, identifies potential gestures through acoustic signals, then detects IMU motion data.
     • Uses an SVM model to distinguish gestures from noise.
  3. Gesture Classification:
     • Employs a K-Nearest Neighbor algorithm (K=1) based on DTW distance for gesture recognition.
  4. User Experiments and Analysis: Collects user behavior data and validates the model's performance in real-world scenarios.

Research Outcomes

• Specific Results:
  • Achieved a 90.9% accuracy rate in recognizing 13 teeth gestures in a laboratory setting.
  • Analyzed the differences between single-sided and dual-sided sensors: dual-sided sensors are crucial for position gesture recognition, while single-sided sensors suffice for distinguishing gesture "types" (single tap, double tap, hold).
  • Maintained approximately 85.3% recognition accuracy even after the device was repositioned.
• Advantages Over Existing Solutions:
  • Low Intrusiveness: Compared to other methods involving sensors placed directly inside the oral cavity, this system is comfortable and easy to use.
  • Cross-Modal Integration: The combination of sound and motion sensing effectively reduces noise interference.
  • Rich Gesture Set: Offers 13 distinct gestures, supporting multi-scenario and multi-purpose applications.
• Experiments and Evaluation:
  • Tested scenarios included stationary postures and dynamic activities (e.g., walking, eating, running), demonstrating excellent classification accuracy and noise filtering performance.
  • Activation gestures (similar to "Hey Siri") did not trigger false positives in simulated real-world environments but exhibited some false negatives.
• Limitations and Future Directions:
  • Limitations:
    • Performance in dynamic scenarios is not fully optimized (e.g., signal interference during user movement).
    • Training models require user-specific customization, and device position changes may affect performance.
    • Sample homogeneity: Participants were limited to young adults aged 21-34, with no exploration of performance in older adults or individuals with disabilities.
  • Future Directions:
    • Enhance robustness in dynamic scenarios and explore advanced machine learning techniques.
    • Improve device design, such as fixing positions to reduce placement deviations.
    • Expand the demographic scope to study applications for aging populations and individuals with mobility impairments.
    • Integrate the system into existing wearable devices (e.g., headphones, glasses) to enhance commercialization potential.

The above is a concise summary of the core content of the paper, clearly presenting the research problem, methods, and outcomes.