FingerGlass: Enhancing Smart Glasses Interaction via Fingerprint Sensing

Research Background and Issues

• Challenges Identified by the Authors:
  Input methods for smart glasses face several limitations. Touchpads suffer from poor precision due to limited space; voice input raises privacy concerns and performs poorly in noisy environments; gesture recognition based on vision or IMU requires high computational costs and may impact battery life. These issues constrain the practicality of smart glasses in daily life.

• Importance of the Problem:
  As the functionality of smart glasses expands, users increasingly demand more intuitive, efficient, and socially acceptable input methods. Developing an input method that meets these requirements is crucial for promoting the widespread adoption of smart glasses in personal and commercial domains.

• Research Motivation and Related Work:
  The authors analyzed the shortcomings of existing input methods (e.g., touchpads, voice recognition, mechanical buttons) and explored novel technologies (e.g., fingerprint sensors, gesture recognition) to enhance the interaction capabilities of smart glasses. Fingerprint sensors were chosen as the research focus due to their mature technology and ease of integration.

Solution

• Method or Solution:
  The authors proposed an interaction technology called FingerGlass, which integrates fingerprint sensors into the temple arms of smart glasses. Users can perform four primary gestures—sliding, scrolling, rotating, and tapping—to interact with the device. Additionally, fingerprint recognition enables differentiation between users or fingers, mapping each gesture to specific device commands.

• Innovations:

  • Combining fingerprint recognition and gesture recognition to create a rich interaction space.
  • Offering a broader input range compared to touchpads and higher privacy and lower computational costs compared to voice, vision, and IMU-based methods.
  • Leveraging mature fingerprint sensor technology to achieve a simplified and comfortable interaction design.

• Implementation Steps and Key Technologies:

  1. Hardware Design: Integrating fingerprint sensors into the temple arms of the glasses to capture fingerprint image sequences for identifying gesture types and finger identity.
  2. Data Preprocessing: Includes image enhancement, noise removal, gesture segmentation, and sequence normalization.
  3. Machine Learning Models: Utilizing lightweight CNN and LSTM networks for fingerprint recognition and gesture classification, respectively.
  4. User Testing and Optimization: Optimizing the interaction experience through task allocation and metric evaluation (e.g., command recognition rate, WPM input speed, and error rate).

Research Outcomes

• Specific Results:
  FingerGlass achieved over 96% accuracy in finger gesture recognition and supported a text input speed of 12.72 words per minute. User feedback indicated that the technology is intuitive, comfortable, and socially acceptable.

• Advantages Over Existing Solutions:

  • Enhanced privacy in interactions: Compared to voice commands and visual gestures, FingerGlass offers a more discreet and unobtrusive input method.
  • Higher recognition accuracy and scalability: Fingerprint recognition enables the same gesture to be mapped to different commands, significantly enriching interaction modes.
  • Superior performance in NASA-TLX stress evaluation: FingerGlass outperformed existing 1D Handwriting input methods across multiple dimensions.

• Experimental or Evaluation Results:
  User testing demonstrated that FingerGlass's text input speed and accuracy improved significantly with usage over time. After extended training, users achieved an input speed of 23.5 words per minute, with an error rate reduced to 3.12%. Additionally, it exhibited lower scores across all dimensions of user workload, such as physical demand, time pressure, and cognitive load.

• Limitations and Future Directions:

  1. The current device is relatively bulky and not suitable for everyday wear. Future work will focus on developing low-power, miniaturized chips for seamless integration with smart glasses.
  2. The prototype lacks visual feedback, increasing the learning curve for users. Future improvements could include integrating language models and displays to optimize input efficiency.
  3. Gesture-to-command mapping may not be optimal for all users, necessitating further exploration of personalization and machine learning optimization.
  4. Research is needed to utilize fingerprint sensors to capture additional finger attributes (e.g., shear force, angle) to enable richer input dimensions.

In summary, FingerGlass provides an efficient, intuitive, and socially acceptable solution for smart glasses interaction design, showcasing significant application potential.