BudsID: Mobile-Ready and Expressive Finger Identification Input for Earbuds

Research Background and Issues

• Identified Problems or Challenges:
  Traditional wireless earbuds are constrained by their small size, lack of visual interfaces, and dynamic usage scenarios (e.g., operating while walking), which limit input methods to a few touch or swipe gestures. Existing interaction technologies for earbuds still lack expressive input capabilities, making it difficult to meet increasingly complex user demands.

• Significance:
  As wireless earbuds expand their functionalities, such as health tracking, biometric authentication, and smart home control, the demand for richer interactive input continues to grow. Developing new interaction methods for earbuds, especially efficient input in dynamic scenarios, can significantly enhance user experience and broaden application contexts.

• Research Motivation and Related Work:
  While previous studies have explored input technologies such as magnetic sensing and finger recognition for wearable devices like smartwatches, no research has yet investigated the potential application of these technologies to earbuds. Existing earbud input methods, such as ear deformation detection and gesture input near the face, have not been tested for performance in dynamic scenarios like walking.

Solution

• Proposed Solution:
  This paper introduces a system called BudsID, which uses a magnetometer embedded in earbuds and a magnetic ring worn on the user's finger to detect touch from different fingers for finger recognition. This input method enables differentiation between touches from various fingers, allowing mapping to multiple functions.

• Innovations:

  1. Adaptation to New Earbud Scenarios: Finger recognition technology is applied to earbuds for the first time, breaking through input limitations in dynamic scenarios.
  2. Superior Classification Performance: The developed deep learning model achieves an accuracy of up to 96.9%.
  3. Support for Multi-Finger Double-Tap Input: Combining consecutive double-tap gestures further enhances input expressiveness, supporting nine distinct input operations.
  4. Hardware Openness: An open-source prototype is designed to enable model operation on low-resource devices.

• Implementation Steps and Key Technologies:

  1. Hardware Prototype Design: A wireless earbud capable of detecting three-dimensional magnetic field changes was constructed using an Arduino Nano BLE Sense board and a magnetometer, paired with a finger ring embedded with a magnet.
  2. Data Collection and Preprocessing: Magnetic field data was sampled and preprocessed, including normalization and data windowing, before being input into the classification model.
  3. Classification Model Design:
     • 1D Convolutional Neural Network (1D-CNN): Processes raw sensor data and features a lightweight architecture suitable for mobile devices.
     • Support Vector Machine (SVM): Utilizes simple statistical features as a basis, offering an alternative for resource-constrained devices.
  4. User Testing: Two studies (single-touch and double-touch) were conducted to validate the technology's accuracy and usability, with results applied to design practical application scenarios.

Research Outcomes

• Specific Results:

  1. User Performance: Experiments show that users can quickly (single-touch duration: 0.98 seconds; double-touch interval: 0.4 seconds) and accurately (error rate: 2.8%-5.6%) complete touch tasks.
  2. Classification Performance:
     • The deep learning classifier for single-finger touch achieved an average accuracy of 96.4%.
     • Classification accuracy for consecutive double-finger touches reached 94.7% (individual models).
     • The SVM classifier performed slightly lower but is suitable for low-resource applications.
  3. Usability and Feedback: BudsID outperformed traditional single-point touch interaction modes in subjective evaluations, achieving higher scores in usability and memorability (e.g., SUS score of 69.25, surpassing the traditional system's 60).

• Advantages over Existing Solutions:

  • Accuracy surpasses similar finger recognition applications on devices like smartwatches (e.g., ultrasound: 93.7%, capacitive touch: 79.4%).
  • Performs exceptionally well in dynamic mobile scenarios and supports highly extensible double-touch gestures.
  • Operates on low-resource microcontroller platforms (e.g., earbud prototypes) with latency suitable for real-time requirements.

• Limitations and Future Directions:

  1. Wearing Comfort: User acceptance of wearing magnetic rings needs improvement. Future exploration could focus on integrating magnets into smaller devices or alternative designs (e.g., magnets attachable to fingernails).
  2. Behavioral Variations: Classification confusion between different fingers (especially middle and ring fingers) may be influenced by arm swing and other behaviors. Optimization of user guidance and device design could address this issue.
  3. Input Extensibility: Currently, only tapping gestures are supported. Future work could incorporate sliding, dragging, and other gestures to enrich input methods.
  4. Low-Resource Optimization: While the lightweight architecture performs well, double-tap classification accuracy (82.4% LOOCV) in new user scenarios still requires improvement.

This research provides a significant pathway for the future development of wireless earbuds, proposing an innovative, accurate, and user-friendly finger recognition input method that lays the foundation for enabling more complex interaction scenarios.