EmoGlass: an End-to-End AI-Enabled Wearable Platform for Enhancing Self-Awareness of Emotional Health

Document Title

EmoGlass: an End-to-End AI-Enabled Wearable Platform for Enhancing Self-Awareness of Emotional Health

Document Information

• Subject Area: Wearable technology and emotion perception, mental health
• Keywords: facial expression detection, emotion perception, mental health, mobile health, wearable devices

Research Background and Issues

• Problems and Challenges:

  • Emotional health issues are often overlooked, leading to potential mental illnesses not being detected in time.
  • Traditional emotion recognition research is mostly confined to laboratory settings, lacking practical applications in daily life.
  • There is a lack of systems designed for end-users that are sustainable and provide personalized emotional regulation suggestions.
  • Existing studies primarily focus on facial expression recognition technology, without fully exploring how wearable technology can enhance users' emotional health.

• Importance:

  • Early detection and intervention in emotional disorders can significantly improve people's health and well-being.
  • Wearable devices, with their low cost, high portability, and user acceptance, offer new solutions for emotional health.

• Research Motivation and Related Work:

  • Emotion perception based on facial expressions has strong theoretical support, but existing devices lack adaptability in complex scenarios.
  • There is a lack of interactive interfaces that provide feedback and suggestions to users, as well as user-centered emotional regulation mechanisms.

Solution

• Proposed Method: Designed EmoGlass, an end-to-end AI-enhanced wearable device, including smart glasses for facial expression detection and a complementary mobile application.

• Innovations:

  1. Hardware Design:
     • Smart glasses integrated with cameras capable of capturing partial facial expressions.
     • Utilized a deep convolutional neural network (ACNN) with attention mechanisms for expression recognition.
  2. Data Iterative Construction:
     • Created three datasets covering controlled laboratory scenarios to complex natural environments (e.g., varying lighting, reassembled frames).
  3. Application Design Improvements:
     • Provided visual feedback on user emotions (e.g., daily/weekly reports).
     • Supported recording activities that triggered emotions and used historical data to assist users in emotional regulation.
     • Educated users on the definition of emotional health and the importance of emotional self-awareness.

• Implementation Steps and Techniques:

  1. Hardware assembly and angle optimization: Used 3D printing to create the glasses frame and employed cameras at specific angles to capture partial facial features.
  2. Data preprocessing and augmentation: Converted images to grayscale, extracted key points, adjusted lighting uniformity, and introduced affine transformations to enhance data diversity.
  3. Model training and transfer: Trained and validated the custom ACNN model sequentially on controlled laboratory data, lighting variation data, and natural facial expression data.
  4. Mobile application design: Developed a React Native app to display reports and provide real-time feedback on emotion detection results.

Research Outcomes

• Experimental Results:

  • In a three-day field study (15 participants, each wearing the device for over 3 hours daily), EmoGlass achieved an average accuracy of 73.0% in detecting seven categories of emotional expressions.
  • Provided users with statistical reports on emotional fluctuations and supported self-reflection and emotion trigger recording functionality.
  • Improved application design received higher user satisfaction feedback, with some users confirming enhanced emotional self-awareness.

• Comparative Advantages:

  • Compared to existing laboratory equipment, EmoGlass is more suitable for daily applications and offers higher ease of use for users.
  • Combining emotion perception with mobile applications effectively enhances users' understanding and control of emotional health.

• Limitations:

  1. The model is highly dependent on individuals, requiring initial calibration, which limits cross-user scalability.
  2. In challenging scenarios (e.g., low light or intense facial movements), the model's accuracy decreases.
  3. Emotion detection relies solely on facial expressions, without integrating other potential indicators (e.g., voice, body movements).

• Future Research Directions:

  1. Hardware Improvements:
     • Optimize power consumption design and explore multi-sensor integration.
     • Modularize the device to adapt to different forms (e.g., hats or accessories).
  2. Model Generalization:
     • Develop user-independent universal models for plug-and-play functionality.
     • Enhance model robustness using synthetic data from virtual cameras and 3D facial mapping technology.
  3. Expanding Emotion Recognition Scope:
     • Explore methods for detecting compound emotions.
     • Integrate richer sensing modalities, such as depth cameras and physiological signals.

Conclusion

EmoGlass is an AI-based wearable platform that provides real-time emotion monitoring and health regulation suggestions, demonstrating the broad potential of wearable devices in the mental health domain. This study is significant for promoting the everyday application of emotion perception technology and early intervention in emotional health issues.