Caught by Surprise, Caught by Culture: Bridging Facial Expression's Recognition and Interpretation of Surprise Across CulturesFacial expressions are powerful signals of human emotion, shaping both human–human and human–computer interaction. As interactive technologies, from adaptive interfaces to emotion-aware agents, become more pervasive, systems are increasingly expected to recognize and respond to users' emotions naturally. But what if a system misreads your face? Such misinterpretation is particularly likely when cultural differences in emotion perception are overlooked. This problem may be compounded by the fact that most facial emotion recognition (FER) models are trained on datasets that reflect the norms of a particular cultural group that assume universality, limiting their reliability in multicultural contexts. Surprise, in particular, is an emotion whose valence can be either positive or negative depending on context, making it a critical case for investigating cultural bias in FER. To address this, we examined how cultural background shapes the recognition and valence interpretation of surprise facial expressions among South Korean (N=36) and American (N=34) participants. Participants labeled 200 facial expressions (surprise and fear), rated their perceived valence, and described personal experiences of surprise. Results show that South Korean-labeled surprise expressions exhibited stronger negative Action Unit (AU) activation and lower valence ratings, whereas American-labeled ones showed more balanced or positive facial cues. Qualitative accounts further revealed that South Koreans framed surprise as tense or socially cautious, while Americans viewed it as open and situationally flexible. These findings bridge recognition and interpretation in cross-cultural emotion research and highlight the need for culturally adaptive FER systems that can interpret ambiguous emotions like surprise more inclusively.2026HJHa Eun Jang et al.Yonsei UniversityEmotion Recognition & DetectionAffective Feedback & Emotion Regulation InterfacesMultilingual & Cross-Cultural Voice InteractionIUI
LingoQ: Bridging the Gap between EFL Learning and Work through AI-Generated Work-Related QuizzesNon-native English speakers performing English-related tasks at work struggle to sustain EFL learning, despite their motivation. Often, study materials are disconnected from their work context. Our formative study revealed that reviewing work-related English becomes burdensome with current systems, especially after work. Although workers rely on LLM-based assistants to address their immediate needs, these interactions may not directly contribute to their English skills. We present LingoQ, an AI-mediated system that allows workers to practice English using quizzes generated from their LLM queries during work. LingoQ leverages these on-the-fly queries using AI to generate personalized quizzes that workers can review and practice on their smartphones. We conducted a three-week deployment study with 28 EFL workers to evaluate LingoQ. Participants valued the quality-assured, work-situated quizzes and constantly engaging with the app during the study. This active engagement improved self-efficacy and led to learning gains for beginners and, potentially, for intermediate learners. Drawing on these results, we discuss design implications for leveraging workers' growing reliance on LLMs to foster proficiency and engagement while respecting work boundaries and ethics.2026YYYeonsun Yang et al.DGISTHuman-LLM CollaborationProgramming Education & Computational ThinkingIntelligent Tutoring Systems & Learning AnalyticsCHI
Modeling User Performance in Multi-Lane Moving-Target AcquisitionModern video games often feature moving target acquisition (MTA) tasks, where users must press a button when a moving target reaches an acquisition line. User performance models in MTA are useful for quantitative skill analysis and computational game level design, but have so far been constructed only for cases where there is a single lane for a target to appear and follow. In this study, the first user performance model is presented and validated for an MTA task with multiple lanes. The model is built as an integration of the existing MTA model and the drift-diffusion model, a model of human decision-making process under time-pressure. In a user study, we showed that the model can fit lane recognition error rates and input timing distributions with significantly higher coefficients of determination ($R^2$) and accuracy than a baseline model.2025JKJonghyun Kim et al.Yonsei UniversityVisualization Perception & CognitionSerious & Functional GamesCHI
Hardware-Embedded Pointing Transfer Function Capable of Canceling OS GainsWhen using indirect pointing devices in modern operating systems (OS), users' perception of the pointing transfer function is easily influenced by the device's hardware or OS-native transfer function settings. This could hinder users from finding and fully adapting to the transfer function that is optimal for them. We propose a novel hardware-embedded transfer function technique that is expected to allow users to consistently experience the desired function even when device hardware or OS settings change. The technique (1) allows users to define the desired function within the device firmware in physical units and (2) enables the firmware to cancel out the influence of OS-native functions and hardware setting perturbations, so that the uploaded function can persist regardless of the external environment. Through technical evaluation including transfer functions of various shapes, we showed that the proposed technique has comparable robustness and accuracy to the conventional approach.2025SKSeonho Kim et al.Yonsei University, Computer Science/College of Computing/Yonsei Esports LabForce Feedback & Pseudo-Haptic WeightCircuit Making & Hardware PrototypingCHI
Effects of Computer Mouse Lift-off Distance Settings in Mouse Lifting ActionThis study investigates the effect of Lift-off Distance (LoD) on a computer mouse, which refers to the height at which a mouse sensor stops tracking when lifted off the surface. Although a low LoD is generally preferred to avoid unintentional cursor movement in mouse lifting (=clutching), especially in first-person shooter games, it may reduce tracking stability. We conducted a psychophysical experiment to measure the perceptible differences between LoD levels and quantitatively measured the unintentional cursor movement error and tracking stability at four levels of LoD while users performed mouse lifting. The results showed a trade-off between movement error and tracking stability at varying levels of LoD. Our findings offer valuable information on optimal LoD settings, which could serve as a guide for choosing a proper mouse device for enthusiastic gamers.2024MKMunjeong Kim et al.Vibrotactile Feedback & Skin StimulationGame UX & Player BehaviorUIST
SERENUS: Alleviating Low-Battery Anxiety Through Real-time, Accurate, and User-Friendly Energy Consumption Prediction of Mobile ApplicationsLow-battery anxiety has emerged as a result of growing dependence on mobile devices, where the anxiety arises when the battery level runs low. While battery life can be extended through power-efficient hardware and software optimization techniques, low-battery anxiety will remain a phenomenon as long as mobile devices rely on batteries. In this paper, we investigate how an accurate real-time energy consumption prediction at the application-level can improve the user experience in low-battery situations. We present Serenus, a mobile system framework specifically tailored to predict the energy consumption of each mobile application and present the prediction in a user-friendly manner. We conducted user studies using Serenus to verify that highly accurate energy consumption predictions can effectively alleviate low-battery anxiety by assisting users in planning their application usage based on the remaining battery life. We summarize requirements to mitigate users’ anxiety, guiding the design of future mobile system frameworks.2024SLSera Lee et al.Privacy by Design & User ControlContext-Aware ComputingNotification & Interruption ManagementUIST
Exploring Intervention Techniques to Alleviate Negative Emotions during Video Content Moderation Tasks as a Worker-centered Task DesignVideos are dynamic and multi-modal compared to other types of content, making automatic filtering difficult, which is why content moderators play a crucial role. However, video content moderators are exposed to more profound emotional labor because videos contain rich visual information, sometimes including even harmful content, such as violent or terrifying scenes. In this work, we explore the effect of six intervention techniques on alleviating negative emotions during video content moderation tasks. We conducted one online crowdsourcing experiment and two controlled user studies to find out that (a) interleaving with positive videos or (b) cartoonization could significantly reduce negative emotions in the moderators. Participants reported that the advantages of these approaches are in helping reduce negative emotions at the time of moderation while existing approaches focus on post-task activities (e.g., relaxation or getting a hobby). We discuss the applicability of our findings to broader tasks, including improvement in intervention techniques.2024DLDokyun Lee et al.Privacy by Design & User ControlOnline Harassment & Counter-ToolsContent Moderation & Platform GovernanceDIS
Find the Bot!: Gamifying Facial Emotion Recognition for Both Human Training and Machine Learning Data CollectionFacial emotion recognition (FER) constitutes an essential social skill for both humans and machines to interact with others. To this end, computer interfaces serve as valuable tools for training individuals to improve FER abilities, while also serving as tools for gathering labels to train FER machine learning datasets. However, existing tools have limitations on the scope and methods of training non-clinical populations and also on collecting labels for machines. In this study, we introduce Find the Bot!, an integrated game that effectively engages the general population to support not only human FER learning on spontaneous expressions but also the collection of reliable judgment-based labels. We incorporated design guidelines from gamification, education, and crowdsourcing literature to engage and motivate players. Our evaluation (N=59) shows that the game encourages players to learn emotional social norms on perceived facial expressions with a high agreement rate, facilitating effective FER learning and reliable label collection all while enjoying gameplay.2024YYYeonsun Yang et al.DGISTGame UX & Player BehaviorGame AccessibilityPrototyping & User TestingCHI
FLUID-IoT : Flexible and Fine-Grained Access Control in Shared IoT Environments via Multi-user UI DistributionThe rapid growth of the Internet of Things (IoT) in shared spaces has led to an increasing demand for sharing IoT devices among multiple users. Yet, existing IoT platforms often fall short by offering an all-or-nothing approach to access control, not only posing security risks but also inhibiting the growth of the shared IoT ecosystem. This paper introduces FLUID-IoT, a framework that enables flexible and granular multi-user access control, even down to the User Interface (UI) component level. Leveraging a multi-user UI distribution technique, FLUID-IoT transforms existing IoT apps into centralized hubs that selectively distribute UI components to users based on their permission levels. Our performance evaluation, encompassing coverage, latency, and memory consumption, affirm that FLUID-IoT can be seamlessly integrated with existing IoT platforms and offers adequate performance for daily IoT scenarios. An in-lab user study further supports that the framework is intuitive and user-friendly, requiring minimal training for efficient utilization.2024SLSunjae Lee et al.KAISTContext-Aware ComputingSmart Home Interaction DesignSmart Home Privacy & SecurityCHI
User Performance in Consecutive Temporal Pointing: An Exploratory StudyA significant amount of research has recently been conducted on user performance in so-called temporal pointing tasks, in which a user is required to perform a button input at the timing required by the system. Consecutive temporal pointing (CTP), in which two consecutive button inputs must be performed while satisfying temporal constraints, is common in modern interactions, yet little is understood about user performance on the task. Through a user study involving 100 participants, we broadly explore user performance in a variety of CTP scenarios. The key finding is that CTP is a unique task that cannot be considered as two ordinary temporal pointing processes. Significant effects of button input method, motor limitations, and different hand use were also observed.2024DLDawon Lee et al.KAISTNotification & Interruption ManagementUser Research Methods (Interviews, Surveys, Observation)CHI
Quantifying Wrist-Aiming Habits with A Dual-Sensor Mouse: Implications for Player Performance and WorkloadComputer mice are widely used today as the primary input device in competitive video games. If a player exhibits more wrist rotation than other players when moving the mouse laterally, the player is said to have stronger wrist-aiming habits. Despite strong public interest, there has been no affordable technique to quantify the extent of a player's wrist-aiming habits and no scientific investigation into how the habits affect player performance and workload. We present a reliable and affordable technique to quantify the extent of a player's wrist-aiming habits using a mouse equipped with two optical sensors (i.e., a dual-sensor mouse). In two user studies, we demonstrate the reliability of the technique and examine the relationship between wrist-aiming habits and player performance or workload. In summary, player expertise and mouse sensitivity significantly impacted wrist-aiming habits; the extent of wrist-aiming showed a positive correlation with upper limb workload.2024DKDonghyeon Kang et al.YONSEI UniversityGame UX & Player BehaviorCHI
DynamicLabels: Supporting Informed Construction of Machine Learning Label Sets with Crowd FeedbackLabel set construction—deciding on a group of distinct labels—is an essential stage in building a supervised machine learning (ML) application, as a badly designed label set negatively affects subsequent stages, such as training dataset construction, model training, and model deployment. Despite its significance, it is challenging for ML practitioners to come up with a well-defined label set, especially when no external references are available. Through our formative study (n=8), we observed that even with the help of external references or domain experts, ML practitioners still need to go through multiple iterations to gradually improve the label set. In this process, there exist challenges in collecting helpful feedback and utilizing it to make optimal refinement decisions. To support informed refinement, we present DynamicLabels, a system that aims to support a more informed label set-building process with crowd feedback. Crowd workers provide annotations and label suggestions to the ML practitioner’s label set, and the ML practitioner can review the feedback through multi-aspect analysis and refine the label set with crowd-made labels. Through a within-subjects study (n=16) using two datasets, we found that DynamicLabels enables better understanding and exploration of the collected feedback and supports a more structured and flexible refinement process. The crowd feedback helped ML practitioners explore diverse perspectives, spot current weaknesses, and shop from crowd-generated labels. Metrics and label suggestions in DynamicLabels helped in obtaining a high-level overview of the feedback, gaining assurance, and spotting surfacing conflicts and edge cases that could have been overlooked.2024JPJeongeon Park et al.AI-Assisted Decision-Making & AutomationCrowdsourcing Task Design & Quality ControlIUI
ModSandbox: Facilitating Online Community Moderation Through Error Prediction and Improvement of Automated RulesDespite the common use of rule-based tools for online content moderation, human moderators still spend a lot of time monitoring them to ensure they work as intended. Based on surveys and interviews with Reddit moderators who use AutoModerator, we identified the main challenges in reducing false positives and false negatives of automated rules: not being able to estimate the actual effect of a rule in advance and having difficulty figuring out how the rules should be updated. To address these issues, we built ModSandbox, a novel virtual sandbox system that detects possible false positives and false negatives of a rule and visualizes which part of the rule is causing issues. We conducted a comparative, between-subject study with online content moderators to evaluate the effect of ModSandbox in improving automated rules. Results show that ModSandbox can support quickly finding possible false positives and false negatives of automated rules and guide moderators to improve them to reduce future errors.2023JSJean Y Song et al.DGISTExplainable AI (XAI)AI-Assisted Decision-Making & AutomationContent Moderation & Platform GovernanceCHI
It is Okay to be Distracted: How Real-time Transcriptions Facilitate Online Meeting with DistractionOnline meetings are indispensable in collaborative remote work environments, but they are vulnerable to distractions due to their distributed and location-agnostic nature. While distraction often leads to a decrease in online meeting quality due to loss of engagement and context, natural multitasking has positive tradeoff effects, such as increased productivity within a given time unit. In this study, we investigate the impact of real-time transcriptions (i.e., full-transcripts, summaries, and keywords) as a solution to help facilitate online meetings during distracting moments while still preserving multitasking behaviors. Through two rounds of controlled user studies, we qualitatively and quantitatively show that people can better catch up with the meeting flow and feel less interfered with when using real-time transcriptions. The benefits of real-time transcriptions were more pronounced after distracting activities. Furthermore, we reveal additional impacts of real-time transcriptions (e.g., supporting recalling contents) and suggest design implications for future online meeting platforms where these could be adaptively provided to users with different purposes.2023SSSeoyun Son et al.KAISTRemote Work Tools & ExperienceNotification & Interruption ManagementCHI
Promptiverse: Scalable Generation of Scaffolding Prompts Through Human-AI Hybrid Knowledge Graph AnnotationOnline learners are hugely diverse with varying prior knowledge, but most instructional videos online are created to be one-size-fits-all. Thus, learners may struggle to understand the content by only watching the videos. Providing scaffolding prompts can help learners overcome these struggles through questions and hints that relate different concepts in the videos and elicit meaningful learning. However, serving diverse learners would require a spectrum of scaffolding prompts, which incurs high authoring effort. In this work, we introduce Promptiverse, an approach for generating diverse, multi-turn scaffolding prompts at scale, powered by numerous traversal paths over knowledge graphs. To facilitate the construction of the knowledge graphs, we propose a hybrid human-AI annotation tool, Grannotate. In our study (N=24), participants produced 40 times more on-par quality prompts with higher diversity, through Promptiverse and Grannotate, compared to hand-designed prompts. Promptiverse presents a model for creating diverse and adaptive learning experiences online.2022YLYoonjoo Lee et al.KAISTIntelligent Tutoring Systems & Learning AnalyticsCrowdsourcing Task Design & Quality ControlCHI
Do We Need a Faster Mouse? Empirical Evaluation of Asynchronicity-Induced JitterIn gaming, accurately rendering input signals on a display is crucial, both spatially and temporally. However, the asynchronicity between the input and output signal frequencies results in unstable responses called "jitter." A recent research modeled this jitter mathematically; however, the effect of jitter on human performance is unknown. In this study, we investigated the empirical effect of asynchronicity-induced jitter using a state-of-the-art high-performance mouse and monitor device. In the first part, perceptual user experience under different jitter levels was examined using the ISO 4120:2004 triangle test protocol, and a jitter of over 0.3 ms could be perceived by sensitive subjects. In the second part, we measured the pointing task performance for different jitter levels using the ISO 9241-9 (i.e., Fitts' law) test, and found that the pointing performance was unaffected up to a jitter of 1 ms. Finally, we recommended display and mouse combinations based on our results, which indicated the need for a higher mouse polling rate than that of the current standard 1000-Hz USB mouse.2021AHAuejin Ham et al.Game UX & Player BehaviorGamification DesignUIST
Secrets of Gosu: Understanding Physical Combat Skills of Professional Players in First-Person ShootersIn first-person shooters (FPS), professional players (a.k.a., Gosu) outperform amateur players. The secrets behind the performance of professional FPS players have been debated in online communities with many conjectures; however, attempts of scientific verification have been limited. We addressed this conundrum through a data-collection study of the gameplay of eight professional and eight amateur players in the commercial FPS Counter-Strike: Global Offensive. The collected data cover behavioral data from six sensors (motion capture, eye tracker, mouse, keyboard, electromyography armband, and pulse sensor) and in-game data (player data and event logs). We examined conjectures in four categories: aiming, character movement, physicality, and device and settings. Only 6 out of 13 conjectures were supported with statistically sufficient evidence.2021EPEunji Park et al.KAISTGame UX & Player BehaviorCHI
Swap: A Replacement-based Text Revision Technique for Mobile DevicesText revision is an important task to ensure the accuracy of text content. Revising text on mobile devices is cumbersome and time-consuming due to the imprecise caret control and the repetitive use of the backspace. We present Swap, a novel replacement-based technique to facilitate text revision on mobile devices. We conducted two user studies to validate the feasibility and the effectiveness of Swap compared to traditional text revision techniques. Results showed that Swap reduced efforts in caret control and repetitive backspace pressing during the text revision process. Most participants preferred to use the replacement-based technique rather than backspace and caret. They also commented that the new technique is easy to learn, and it makes text revision rapid and intuitive.2020YLYang Li et al.Kochi University of TechnologyContext-Aware ComputingPrototyping & User TestingCHI
Optimal Sensor Position for a Computer MouseComputer mice have their displacement sensors in various locations (center, front, and rear). However, there has been little research into the effects of sensor position or on engineering approaches to exploit it. This paper first discusses the mechanisms via which sensor position affects mouse movement and reports the results from a study of a pointing task in which the sensor position was systematically varied. Placing the sensor in the center turned out to be the best compromise: improvements over front and rear were in the 11-14% range for throughput and 20--23% for path deviation. However, users varied in their personal optima. Accordingly, variable-sensor-position mice are then presented, with a demonstration that high accuracy can be achieved with two static optical sensors. A virtual sensor model is described that allows software-side repositioning of the sensor. Individual-specific calibration should yield an added 4% improvement in throughput over the default center position.2020SKSunjun Kim et al.Aalto University & Daegu Gyeongbuk Institute of Science and TechnologyContext-Aware ComputingPrototyping & User TestingCHI