Smiles, Frowns, and Everything In Between: Understanding User Experiences with Emotion Tracking in Daily Life through Facial Expression Recognition

Emotion tracking is commonly practiced for mental wellbeing. In recent years, increasing attention is turned to automatic emotion tracking, including technologies based on Facial Expression Recognition (FER). However, work on FER based tracking so far has mainly focused on accuracy improvement, and little is on how it works in everyday lives. In this paper, we developed an FER-based emotion tracking web application, EmoAction, and used it as a technology probe to explore participants' experiences and perceptions of using FER-based emotion tracking in daily life. A field study with 9 participants trying it over 14 days reveals how such an emotion tracking tool enables people to gain new awareness and new perspectives of their emotions, and identifies several potential usages of such an automatic tracking tool. We then discuss the implications for future design, including providing real-time non-judgmental feedback, prioritizing triggering reflection over seeking maximum precision, and enabling social sharing with respect for privacy.