Caught by Surprise, Caught by Culture: Bridging Facial Expression's Recognition and Interpretation of Surprise Across Cultures

Facial expressions are powerful signals of human emotion, shaping both human–human and human–computer interaction. As interactive technologies, from adaptive interfaces to emotion-aware agents, become more pervasive, systems are increasingly expected to recognize and respond to users' emotions naturally. But what if a system misreads your face? Such misinterpretation is particularly likely when cultural differences in emotion perception are overlooked. This problem may be compounded by the fact that most facial emotion recognition (FER) models are trained on datasets that reflect the norms of a particular cultural group that assume universality, limiting their reliability in multicultural contexts. Surprise, in particular, is an emotion whose valence can be either positive or negative depending on context, making it a critical case for investigating cultural bias in FER. To address this, we examined how cultural background shapes the recognition and valence interpretation of surprise facial expressions among South Korean (N=36) and American (N=34) participants. Participants labeled 200 facial expressions (surprise and fear), rated their perceived valence, and described personal experiences of surprise. Results show that South Korean-labeled surprise expressions exhibited stronger negative Action Unit (AU) activation and lower valence ratings, whereas American-labeled ones showed more balanced or positive facial cues. Qualitative accounts further revealed that South Koreans framed surprise as tense or socially cautious, while Americans viewed it as open and situationally flexible. These findings bridge recognition and interpretation in cross-cultural emotion research and highlight the need for culturally adaptive FER systems that can interpret ambiguous emotions like surprise more inclusively.