Rethinking Teaching Evaluation Reports: Designing AI-transformed Student Feedback for Instructor EngagementStudent evaluations of teaching (SETs) represent a valuable yet often underutilized resource, as many instructors struggle with the substantial time, cognitive, and emotional demands of processing this feedback effectively. While these evaluations contain crucial insights into students' learning experiences that could enhance instruction, their potential remains largely untapped. Our work explores how to redesign SET reports using language models (LMs) to distill, highlight, and present student feedback in more engaging and actionable ways. We systematically explored a $4 \times 4$ strategy-presentation design space, creating six representative mock-ups that integrate different analytical strategies with various presentation formats. Through interviews with 16 post-secondary instructors, we learned how and when they engage with current SETs, and how they would perceive and use the LM-augmented redesigned SET mock-ups. Our findings revealed that instructors' preferences for different redesigns aligned with distinct goals: whether improving their teaching practices, gaining quick insights into their teaching effectiveness, or preparing summative teaching performance reports. These findings shed light on new opportunities for designing dynamic SET systems where AI can adaptively process and present feedback based on instructors' specific needs and contexts.2025RSRuoxi Shang et al.Technology Use in Higher EducationCSCW
EmoShortcuts: Emotionally Expressive Body Augmentation for Social Mixed Reality AvatarsWe present EmoShortcuts, a novel social Mixed Reality (MR) framework that enhances emotional expression by dynamically augmenting avatar body gestures to reflect users’ emotional states. While social MR enables immersive remote interactions through avatars, conveying emotions remains challenging due to limitations in head-mounted display (HMD) tracking (e.g., missing lower-body movements like stomping or defensive postures) and users' tendency to deprioritize nonverbal expression during multitasking. EmoShortcuts addresses these challenges by introducing an augmentation framework that generates expressive body gestures even when users' physical movements are restricted. We conducted a formative study with 12 participants to identify key challenges in emotional expression and explore user preferences for AI-assisted gesture augmentation. Based on these insights, we designed an interface that enables adaptive gesture augmentation and allows for both pre-set and real-time user control. Through an extensive user study (n = 16), our findings show that EmoShortcuts significantly improves emotion expression accuracy and reduces cognitive workload, demonstrating its potential for more immersive and emotionally rich virtual communication.2025HSJinwook Seo et al.Mixed Reality WorkspacesIdentity & Avatars in XRUIST
Cracking Aegis: An Adversarial LLM-based Game for Raising Awareness of Vulnerabilities in Privacy ProtectionTraditional methods for raising awareness of privacy protection often fail to engage users or provide hands-on insights into how privacy vulnerabilities are exploited. To address this, we incorporate an adversarial mechanic in the design of the dialogue-based serious game Cracking Aegis. Leveraging LLMs to simulate natural interactions, the game challenges players to impersonate characters and extract sensitive information from an AI agent, Aegis. A user study (n=22) revealed that players employed diverse deceptive linguistic strategies, including storytelling and emotional rapport, to manipulate Aegis. After playing, players reported connecting in-game scenarios with real-world privacy vulnerabilities, such as phishing and impersonation, and expressed intentions to strengthen privacy control, such as avoiding oversharing personal information with AI systems. This work highlights the potential of LLMs to simulate complex relational interactions in serious games, while demonstrating how an adversarial strategy provides a unique perspective in designing for social good, particularly in privacy protection.2025JFJiaying Fu et al.Serious & Functional GamesPrivacy Perception & Decision-MakingDark Patterns RecognitionDIS
Enhancing Deliberativeness: Evaluating the Impact of Multimodal Reflection NudgesNudging participants with text-based reflective nudges enhances deliberation quality on online deliberation platforms. The effectiveness of multimodal reflective nudges, however, remains largely unexplored. Given the multi-sensory nature of human perception, incorporating diverse modalities into self-reflection mechanisms has the potential to better support various reflective styles. This paper explores how presenting reflective nudges of different types (direct: persona and indirect: storytelling) in different modalities (text, image, video and audio) affects deliberation quality. We conducted two user studies with 20 and 200 participants respectively. The first study identifies the preferred modality for each type of reflective nudges, revealing that text is most preferred for persona and video is most preferred for storytelling. The second study assesses the impact of these modalities on deliberation quality. Our findings reveal distinct effects associated with each modality, providing valuable insights for developing more inclusive and effective online deliberation platforms.2025SYShunYi Yeo et al.Singapore University of Technology and DesignParticipatory DesignInteractive Narrative & Immersive StorytellingCHI
Prompting an Embodied AI Agent: How Embodiment and Multimodal Signaling Affects Prompting BehaviourCurrent voice agents wait for a user to complete their verbal instruction before responding; yet, this is misaligned with how humans engage in everyday conversational interaction, where interlocutors use multimodal signaling (e.g. nodding, grunting, or looking at referred to objects) to ensure conversational grounding. We designed an embodied VR agent that exhibits multimodal signaling behaviors in response to situated prompts, by turning its head, or by visually highlighting objects being discussed or referred to. We explore how people prompt this agent to design and manipulate the objects in a VR scene. Through a Wizard of Oz study, we found that participants interacting with an agent that indicated its understanding of spatial and action references were able to prevent errors 30% of the time, and were more satisfied and confident in the agent's abilities. These findings underscore the importance of designing multimodal signalling communication techniques for future embodied agents.2025TZTianyi Zhang et al.Singapore Management UniversityFull-Body Interaction & Embodied InputVoice User Interface (VUI) DesignSocial & Collaborative VRCHI
"Ronaldo's a poser!": How the Use of Generative AI Shapes Debates in Online ForumsOnline debates can enhance critical thinking but may escalate into hostile attacks. As humans are increasingly reliant on Generative AI (GenAI) in writing tasks, we need to understand how people utilize GenAI in online debates. To examine the patterns of writing behavior while making arguments with GenAI, we created an online forum for soccer fans to engage in turn-based and free debates in a post format with the assistance of ChatGPT, arguing on the topic of "Messi vs Ronaldo". After 13 sessions of two-part study and semi-structured interviews with 39 participants, we conducted content and thematic analyses to integrate insights from interview transcripts, ChatGPT records, and forum posts. We found that participants prompted ChatGPT for aggressive responses, created posts with similar content and logical fallacies, and sacrificed the use of ChatGPT for better human-human communication. This work uncovers how polarized forum members work with GenAI to engage in debates online.2025YZYuhan Zeng et al.City University of Hong Kong, Department of Computer ScienceGenerative AI (Text, Image, Music, Video)Social Platform Design & User BehaviorMisinformation & Fact-CheckingCHI
“I can run at night!": Using Augmented Reality to Support Nighttime Guided Running for Low-vision RunnersDark environment challenges low-vision (LV) individuals to engage in running by following sighted guide—a Caller-style guided running—due to insufficient illumination, because it prevents them from using their residual vision to follow the guide and be aware about their environment. We design, develop, and evaluate RunSight, an augmented reality (AR)-based assistive tool to support LV individuals to run at night. RunSight combines see-through HMD and image processing to enhance one's visual awareness of the surrounding environment (e.g., potential hazard) and visualize the guide's position with AR-based visualization. To demonstrate RunSight's efficacy, we conducted a user study with 8 LV runners. The results showed that all participants could run at least 1km (mean = 3.44 km) using RunSight, while none could engage in Caller-style guided running without it. Our participants could run safely because they effectively synthesized RunSight-provided cues and information gained from runner-guide communication.2025YAYuki Abe et al.Hokkaido University, Human-Computer Interaction LabAR Navigation & Context AwarenessVisual Impairment Technologies (Screen Readers, Tactile Graphics, Braille)CHI
Cyberoception: Finding A Painlessly-Measurable New Sense In The Cyberworld Towards Emotion-awareness In ComputingIn Affective computing, recognizing users' emotions accurately is the basis of affective human–computer interaction. Understanding users' interoception contributes to a better understanding of individually different emotional abilities, which is essential for achieving inter-individually accurate emotion estimation. However, existing interoception measurement methods, such as the heart rate discrimination task, have several limitations, including their dependence on a well-controlled laboratory environment and precision apparatus, making monitoring users' interoception challenging. This study aims to determine other forms of data that can explain users' interoceptive or similar states in their real-world lives and propose a novel hypothetical concept "cyberoception," a new sense (1) which has properties similar to interoception in terms of the correlation with other emotion-related abilities, and (2) which can be measured only by the sensors embedded inside commodity smartphone devices in users' daily lives. Results from a 10-day-long in-lab/in-the-wild hybrid experiment reveal a specific cyberoception type "Turn On." (users' subjective sensory perception about the frequency of turning-on behavior on their smartphones)2025TOTadashi Okoshi et al.Keio University, Faculty of Environment and Information StudiesBrain-Computer Interface (BCI) & NeurofeedbackBiosensors & Physiological MonitoringCHI
SmarTeeth: Augmenting Manual Toothbrushing with In-ear MicrophonesImproper toothbrushing practices persist as a primary cause of oral health issues such as tooth decay and gum disease. Despite the availability of high-end electric toothbrushes that offer some guidance, manual toothbrushes remain widely used due to their simplicity and convenience. We present SmarTeeth, an earable-based toothbrushing monitoring system designed to augment manual toothbrushing with functionalities typically offered only by high-end electric toothbrushes, such as brushing surface tracking. The underlying idea of SmarTeeth is to leverage in-ear microphones on earphones to capture toothbrushing sounds transmitted through the oral cavity to ear canals through facial bones and tissues. The distinct propagation paths of brushing sounds from various dental locations to each ear canal provide the foundational basis for our methods to accurately identify different brushing locations. By extracting customized features from these sounds, we can detect brushing locations using a deep-learning model. With only one registration session (~2 mins) for a new user, the average accuracy is 92.7% for detecting six regions and 75.6% for sixteen tooth surfaces. With three registration sessions (~6 mins), the performance can be boosted to 98.8% and 90.3% for six-region and sixteen-surface tracking, respectively. A key advantage of using earphones for monitoring is that they provide natural auditory feedback to alert users when they are overbrushing or underbrushing. Comprehensive evaluation validates the effectiveness of SmarTeeth under various conditions (different users, brushes, orders, noise, etc.), and the feedback from the user study (N=13) indicates that users found the system highly useful (6.0/7.0) and reported a low workload (2.5/7.0) while using it. Our findings suggest that SmarTeeth could offer a scalable and effective solution to improve oral health globally by providing manual toothbrush users with advanced brushing monitoring capabilities.2025QYQiang Yang et al.University of Cambridge, Department of Computer Science and TechnologyBiosensors & Physiological MonitoringElectronic Textiles (E-textiles)Context-Aware ComputingCHI
W4-Groups: Modeling the Who, What, When and Where of Group Behavior via Mobility SensingHuman social interactions occur in group settings of varying sizes and locations, depending on the type of social activity. The ability to distinguish group formations based on their purposes transforms how group detection mechanisms function. Not only should such tools support the effective detection of serendipitous encounters, but they can derive categories of relation types among users. Determining who is involved, what activity is performed, and when and where the activity occurs are critical to understanding group processes in greater depth, including supporting goal-oriented applications (e.g., performance, productivity, and mental health) that require sensing social factors. In this work, we propose W4-Groups that captures the functional perspective of variability and repeatability when automatically constructing short-term and long-term groups via multiple data sources (e.g., WiFi and location check-in data). We design and implement W4-Groups to detect and extract all four group features who-what-when-where from the user’s daily mobility patterns. We empirically evaluate the framework using two real-world WiFi datasets and a location check-in dataset, yielding an average of 92% overall accuracy, 96% precision, and 94% recall. Further, we supplement two case studies to demonstrate the application of W4-Groups for next-group activity prediction and analyzing changes in group behavior at a longitudinal scale, exemplifying short-term and long-term occurrences.2024AAAkanksha Atrey et al.Session 3f: Enhancing Virtual Presence and InteractionCSCW
ClearSpeech: Improving Voice Quality of Earbuds Using Both In-Ear and Out-Ear MicrophonesMa 等人提出 ClearSpeech 系统，结合耳塞的内耳和外耳双麦克风进行语音增强，在嘈杂环境中可将语音质量提升 40%，显著改善真无线耳塞的通话体验。2024DMDong Ma et al.Haptic WearablesVoice User Interface (VUI) DesignUbiComp
BreathPro: Monitoring Breathing Mode during Running with EarablesHu 等人开发 BreathPro 系统，利用耳穿戴设备传感器实时监测跑步时的呼吸模式，为运动健康监测提供新方案。2024CHChangshuo Hu et al.Fitness Tracking & Physical Activity MonitoringBiosensors & Physiological MonitoringUbiComp
Conversational Localization: Indoor Human Localization through Intelligent ConversationSheshadri 等人提出 Conversational Localization 方法，通过智能对话交互实现室内人体定位。2024SSSmitha Sheshadri et al.Multilingual & Cross-Cultural Voice InteractionAR Navigation & Context AwarenessUbiComp
Predicting the Limits: Tailoring Unnoticeable Hand Redirection Offsets in Virtual Reality to Individuals’ Perceptual BoundariesMany illusion and interaction techniques in Virtual Reality (VR) rely on Hand Redirection (HR), which has proved to be effective as long as the introduced offsets between the position of the real and virtual hand do not noticeably disturb the user experience. Yet calibrating HR offsets is a tedious and time-consuming process involving psychophysical experimentation, and the resulting thresholds are known to be affected by many variables---limiting HR's practical utility. As a result, there is a clear need for alternative methods that allow tailoring HR to the perceptual boundaries of individual users. We conducted an experiment with 18 participants combining movement, eye gaze and EEG data to detect HR offsets Below, At, and Above individuals' detection thresholds. Our results suggest that we can distinguish HR At and Above from no HR. Our exploration provides a promising new direction with potentially strong implications for the broad field of VR illusions.2024MFMartin Feick et al.Full-Body Interaction & Embodied InputEye Tracking & Gaze InteractionBrain-Computer Interface (BCI) & NeurofeedbackUIST
GradualReality: Enhancing Physical Object Interaction in Virtual Reality via Interaction State-Aware BlendingWe present GradualReality, a novel interface enabling a Cross Reality experience that includes gradual interaction with physical objects in a virtual environment and supports both presence and usability. Daily Cross Reality interaction is challenging as the user's physical object interaction state is continuously changing over time, causing their attention to frequently shift between the virtual and physical worlds. As such, presence in the virtual environment and seamless usability for interacting with physical objects should be maintained at a high level. To address this issue, we present an Interaction State-Aware Blending approach that (i) balances immersion and interaction capability and (ii) provides a fine-grained, gradual transition between virtual and physical worlds. The key idea includes categorizing the flow of physical object interaction into multiple states and designing novel blending methods that offer optimal presence and sufficient physical awareness at each state. We performed extensive user studies and interviews with a working prototype and demonstrated that GradualReality provides better Cross Reality experiences compared to baselines.2024HSHyunA Seo et al.Eye Tracking & Gaze InteractionMixed Reality WorkspacesImmersion & Presence ResearchUIST
Desk2Desk: Optimization-based Mixed Reality Workspace Integration for Remote Side-by-side CollaborationMixed Reality enables hybrid workspaces where physical and virtual monitors are adaptively created and moved to suit the current environment and needs. However, in shared settings, individual users’ workspaces are rarely aligned and can vary significantly in the number of monitors, available physical space, and workspace layout, creating inconsistencies between workspaces which may cause confusion and reduce collaboration. We present Desk2Desk, an optimization-based approach for remote collaboration in which the hybrid workspaces of two collaborators are fully integrated to enable immersive side-by-side collaboration. The optimization adjusts each user’s workspace in layout and number of shared monitors and creates a mapping between workspaces to handle inconsistencies between workspaces due to physical constraints (e.g. physical monitors). We show in a user study how our system adaptively merges dissimilar physical workspaces to enable immersive side-by-side collaboration, and demonstrate how an optimization-based approach can effectively address dissimilar physical layouts.2024LSLudwig Sidenmark et al.Mixed Reality WorkspacesDistributed Team CollaborationUIST
How People Prompt Generative AI to Create Interactive VR ScenesGenerative AI tools can provide people with the ability to create virtual environments and scenes with natural language prompts. Yet, \textit{how} people will formulate such prompts is unclear---particularly when they inhabit the environment that they are designing. For instance, it is likely that a person might say, ``Put a chair here,'' while pointing at a location. If such linguistic and embodied features are common to people's prompts, we need to tune models to accommodate them. In this work, we present a Wizard of Oz elicitation study with 22 participants, where we studied people's implicit expectations when verbally prompting such programming agents to create interactive VR scenes. Our findings show when people prompted the agent, they had several implicit expectations of these agents: (1) they should have an embodied knowledge of the environment; (2) they should understand embodied prompts by users; (3) they should recall previous states of the scene and the conversation, and that (4) they should have a commonsense understanding of objects in the scene. Further, we found that participants prompted differently when they were prompting \textit{in situ} (i.e. within the VR environment) versus \textit{ex situ} (i.e. viewing the VR environment from the outside). To explore how these lessons could be applied, we designed and built Ostaad, a conversational programming agent that allows non-programmers to design interactive VR experiences that they inhabit. Based on these explorations, we outline new opportunities and challenges for conversational programming agents that create VR environments.2024SMSetareh Aghel Manesh et al.Social & Collaborative VRGenerative AI (Text, Image, Music, Video)Human-LLM CollaborationDIS
VAID: Indexing View Designs in Visual Analytics SystemVisual analytics (VA) systems have been widely used in various application domains. However, VA systems are complex in design, which imposes a serious problem: although the academic community constantly designs and implements new designs, the designs are difficult to query, understand, and refer to by subsequent designers. To mark a major step forward in tackling this problem, we index VA designs in an expressive and accessible way, transforming the designs into a structured format. We first conducted a workshop study with VA designers to learn user requirements for understanding and retrieving professional designs in VA systems. Thereafter, we came up with an index structure VAID to describe advanced and composited visualization designs with comprehensive labels about their analytical tasks and visual designs. The usefulness of VAID was validated through user studies. Our work opens new perspectives for enhancing the accessibility and reusability of professional visualization designs.2024LYLu Ying et al.Zhejiang UniversityInteractive Data VisualizationVisualization Perception & CognitionCHI
SwapVid: Integrating Video Viewing and Document Exploration with Direct ManipulationVideos accompanied by documents---\textit{document-based videos}---enable presenters to share contents beyond videos and audience to use them for detailed content comprehension. However, concurrently exploring multiple channels of information could be taxing. We propose SwapVid, a novel interface for viewing and exploring document-based videos. SwapVid seamlessly integrates a video and a document into a single view and lets the content behaves as both video and a document; it adaptively switches a document-based video to act as a video or a document upon direct manipulation (\textit{e.g.,} scrolling the document, manipulating the video timeline). We conducted a user study with twenty participants, comparing SwapVid to a side-by-side video/document views. Results showed that our interface reduces time and physical workload when exploring slide-based documents based on video referencing. Based on the study findings, we extended SwapVid with additional functionalities and demonstrated that it further extends the practical capabilities.2024TMTaichi Murakami et al.Tohoku UniversityInteractive Data VisualizationData StorytellingContext-Aware ComputingCHI
The Impact of Avatar Completeness on Embodiment and the Detectability of Hand Redirection in Virtual RealityTo enhance interactions in VR, many techniques introduce offsets between the virtual and real-world position of users’ hands. Nevertheless, such hand redirection (HR) techniques are only effective as long as they go unnoticed by users—not disrupting the VR experience. While several studies consider how much unnoticeable redirection can be applied, these focus on mid-air floating hands that are disconnected from users’ bodies. Increasingly, VR avatars are embodied as being directly connected with the user’s body, which provide more visual cue anchoring, and may therefore reduce the unnoticeable redirection threshold. In this work, we studied more complete avatars and their effect on the sense of embodiment and the detectability of HR. We found that higher avatar completeness increases embodiment, and we provide evidence for the absence of practically relevant effects on the detectability of HR.2024MFMartin Feick et al.DFKI, Saarland Informatics CampusForce Feedback & Pseudo-Haptic WeightFull-Body Interaction & Embodied InputIdentity & Avatars in XRCHI