TexonMask: Facial Expression Recognition Using Textile Electrodes on Commodity Facemasks

Title of the Paper

TexonMask: Facial Expression Recognition Using Textile Electrodes on Commodity Facemasks

Paper Information

• Research Area: Wearable Devices, Embedded Human-Computer Interaction, Emotion Recognition
• Keywords: Facial Expression Recognition, Embedded Interaction, Capacitive Sensing, Textile Electrodes, Facemasks, Wearable Devices, Edge Computing

Research Background and Problem Statement

• Problems and Challenges:

  1. Wearing facemasks obstructs the observation of facial expressions, weakening emotional communication.
  2. Facial occlusion reduces trust and empathy in communication, with even more pronounced negative impacts on individuals with cognitive or hearing impairments.
  3. Existing facial expression recognition methods (e.g., cameras or adhesive electrodes) either raise privacy concerns or are unsuitable for daily use.

• Significance: Facial expressions are a crucial element of non-verbal communication. Especially in the context of the pandemic, effectively conveying emotions while wearing facemasks has become increasingly important.

• Research Motivation: Addressing the limitations of traditional methods, the study aims to develop a lightweight, user-friendly wearable device for daily use that enhances facial expression transmission and supports related human-computer interaction applications.

Solution

• Core Method: Embedding a textile electrode array into standard facemasks to capture facial muscle movements using capacitive sensing technology, combined with edge computing and lightweight machine learning models for expression recognition.

• Innovations:

  1. Seamlessly integrating electrode design with commercial facemasks without compromising their appearance or filtration performance.
  2. Supporting real-time facial expression recognition with an accuracy of over 90%, while enabling operation on edge devices with low computational resource requirements.
  3. Introducing the "LiveEmoji" application for both face-to-face and online interactions to enhance social experiences.

• Implementation Steps:

  1. Electrode Design and Installation: Arranging an 8×15 matrix of textile electrodes on the mask surface and manually stitching them.
  2. Hardware Setup: Utilizing a microcontroller and capacitive multi-touch module for signal acquisition.
  3. Signal Processing and Model Training: Normalizing background signals and training a linear Support Vector Machine (SVM) classifier, including personalized and cross-user models.
  4. Application Demonstration:
     • In online communication, displaying real-time expressions via LiveEmoji on screens.
     • In face-to-face interactions, showing expressions through an LED matrix worn around the neck.

Research Outcomes

• Specific Results:

  1. Using personalized models (requiring only six data points per expression for training), achieving an accuracy of 90% with stable performance across multiple uses.
  2. Increasing training data further improves accuracy to 95%.

• Comparative Advantages:

  1. Avoiding privacy issues and constraints of specific lighting conditions compared to camera-based solutions.
  2. Compared to traditional sensors, TexonMask is flexible, lightweight, comfortable for daily wear, and washable.

• Experiments and Evaluation:

  1. Two rounds of user studies validated the robustness of personalized models, though cross-user classifier accuracy (~70%) remains suboptimal and requires improvement.
  2. Technical evaluations demonstrated stable system performance across varying electrode densities and sensing scenarios.

• Limitations and Future Directions:

  1. Personalized training requires users to provide a small amount of training data (albeit minimal).
  2. Facial expression recognition exhibits significant variability across users (necessitating broader user diversity in training).
  3. Future explorations:
     • Integrating deep learning to enhance model performance.
     • Investigating continuous expression analysis for implicit emotion recognition.
     • Extending electrode applications to other daily wearables (e.g., headphones, headbands).
     • Improving electrode manufacturing processes to support mass production and larger-scale user testing.

This study provides valuable technological and application insights into enhancing emotional communication and embedded interaction, showcasing innovative potential for post-pandemic human-computer interaction technologies.