Beyond Technical Metrics: Understanding the Gap Between AI Performance and Deaf User Experience in Chinese Natural Sign Language GenerationCurrent sign language generation relies on technical metrics that often overlook capturing actual Deaf user comprehension. This is particularly challenging for Chinese Natural Sign Language (CNSL) with its scene-dependent expressions and spatial grammar. We assembled a scene-aware CNSL generation prototype using established components to serve as a controlled evaluation stimulus. In partnership with four Deaf co-researchers, we developed a seven-dimensional evaluation framework combining objective comprehension tests with validated subjective measures. An evaluation with 24 Deaf participants revealed that while technical metrics indicated success, objective comprehension tests showed lower comprehension and higher cognitive load, particularly for complex spatial grammar. These results demonstrate that current evaluation methods do not align with user needs. Our framework shifts from computer benchmarking toward human-centered assessment, prioritizing comprehension, cognitive load, and cultural authenticity. The findings underscore the importance of creating accessibility technology with, rather than for, Deaf communities.2026YLYang Liu et al.Jilin UniversityVisual Impairment Technologies (Screen Readers, Tactile Graphics, Braille)Deaf & Hard-of-Hearing Support (Captions, Sign Language, Vibration)Voice AccessibilityCHI
Image-based Tactile Rendering Method Using Frequency Modulation under the Weber Fractions Constraint by ElectrovibrationThe tactile display of visual images on touchscreens improves the immersion in multimodal applications. Tactile rendering methods considering discrimination thresholds highlight the perceptual differences of friction variations. In this paper, we present a tactile rendering method using frequency modulation of square wave driven signal under the Weber Fractions (WFs) constraint by electrovibration. Firstly, we report the frequency WFs of square wave under conditions of rising frequency (RF) and falling frequency (FF). Then, we determine the variation of the WFs affected by texel length, sliding speed and applied voltage. Finally, we propose the image-based tactile rendering method. The results indicate that the WFs increase from 0.18 and 0.16 to 0.35 and 0.24 in the range of <50Hz and remain constant in 50-500Hz under RF and FF, respectively. Sliding speed has a significant effect on the WFs under RF. And the rendering fidelity of proposed method has a significantly increase by 17.9% in tactile display.2025FHFei Hong et al.Vibrotactile Feedback & Skin StimulationUIST
Ink Restorer: Virtual Restoration of Ancient Chinese Paintings Inheriting Traditional Restoration ProcessesThe restoration of ancient Chinese paintings plays an essential role in protection and inheritance of Asian culture. A traditional restoration process consists of four stages: Xi (washing), Jie (separating), Bu (mending), and Quan (completing). However, it is difficult for the public to experience this process due to high professional requirement and time consumption. We conduct a questionnaire survey and interview experts in our formative study. The questionnaire result shows the public express strong interest in virtual restoration. Experts believe virtual restoration is an experience valuable for the public. We introduce Ink-Restorer, a tool designed for experiencing virtual restoration for ancient paintings. Its design follows the traditional restoration process, and it adopts image segmentation and generation techniques to simplify detailed restoration for users. We recruit 60 users to evaluate Ink-Restorer and invite experts to evaluate restoration results. Ink-Restorer significantly improves user experience, cultural understanding, and restoration quality.2025YZYing Zhang et al.School of Software Technology, Zhejiang UniversityMuseum & Cultural Heritage DigitizationFood Culture & Food InteractionCHI
Community Fact-Checks Trigger Moral Outrage in Replies to Misleading Posts on Social MediaDisplaying community fact-checks is a promising approach to reduce engagement with misinformation on social media. However, how users respond to misleading content emotionally after community fact-checks are displayed on posts is unclear. Here, we employ quasi-experimental methods to causally analyze changes in sentiments and (moral) emotions in replies to misleading posts following the display of community fact-checks. Our evaluation is based on a large-scale panel dataset comprising N=2,225,260 replies across 1841 source posts from X's Community Notes platform. We find that informing users about falsehoods through community fact-checks significantly increases negativity (by 7.3%), anger (by 13.2%), disgust (by 4.7%), and moral outrage (by 16.0%) in the corresponding replies. These results indicate that users perceive spreading misinformation as a violation of social norms and that those who spread misinformation should expect negative reactions once their content is debunked. We derive important implications for the design of community-based fact-checking systems.2025YCYuwei Chuai et al.University of Luxembourg, SnTMisinformation & Fact-CheckingAlgorithmic Fairness & BiasCHI
PDFChatAnnotator: A Human-LLM Collaborative Multi-Modal Data Annotation Tool for PDF-Format CatalogsThe document contains substantial unannotated data, necessitating extensive manual labeling efforts. To address this issue, we introduce PDFChatAnnotator, a human-LLM collaborative tool to collect multi-modal data from PDF catalogs. Initially, PDFChatAnnotator automatically employs our proposed multi-modal binding rules to link related data from different modalities and harnesses the information extraction capabilities of large language models (LLMs) to extract specific information from text descriptions. Furthermore, the tool empowers users to guide and refine the LLM's annotations. During the annotation process, users can influence the LLM through multiple rounds of communication and example establishment via the provided interfaces. To assess the effectiveness of PDFChatAnnotator's techniques, we conducted a technical evaluation using three catalogs with typical layouts as experimental data. The results showed that all accuracy rates for multi-modal binding exceeded 90%, and both the proposed "example establishment" and "interactive adjustment of requirements" contributed to enhanced accuracy rates.2024YTYi Tang et al.Human-LLM CollaborationPrototyping & User TestingIUI
SpaceEditing: A Latent Space Editing Interface for Integrating Human Knowledge into Deep Neural NetworksHuman-centered AI aims to bridge the gap between machine decision-making and human understanding. However, even for classification tasks where deep neural networks have achieved superb performance, there are currently few methods that link humans and AI well, especially on domain-specific tasks. In this paper, we propose SpaceEditing, a 2D spatial layout tool that enables human users to interact with the latent space of deep neural networks. During the interaction process, the tool's algorithm automatically processes user movements and feedback into the network to learn from user-modified information. We evaluate SpaceEditing with three case studies: (1) an archaeology researcher uses a bronze dataset; (2) a deep learning researcher uses a garbage classification dataset; (3) six deep learning beginners use a head pose dataset. The experimental results demonstrate the effectiveness of our tool in integrating human knowledge and improving network performance.2024JWJiafu Wei et al.Explainable AI (XAI)Algorithmic Transparency & AuditabilityPrivacy by Design & User ControlIUI
I Know Your Intent: Graph-enhanced Intent-aware User Device Interaction Prediction via Contrastive Learning"With the booming of smart home market, intelligent Internet of Things (IoT) devices have been increasingly involved in home life. To improve the user experience of smart homes, some prior works have explored how to use machine learning for predicting interactions between users and devices. However, the existing solutions have inferior User Device Interaction (UDI) prediction accuracy, as they ignore three key factors: routine, intent and multi-level periodicity of human behaviors. In this paper, we present SmartUDI, a novel accurate UDI prediction approach for smart homes. First, we propose a Message-Passing-based Routine Extraction (MPRE) algorithm to mine routine behaviors, then the contrastive loss is applied to narrow representations among behaviors from the same routines and alienate representations among behaviors from different routines. Second, we propose an Intent-aware Capsule Graph Attention Network (ICGAT) to encode multiple intents of users while considering complex transitions between different behaviors. Third, we design a Cluster-based Historical Attention Mechanism (CHAM) to capture the multi-level periodicity by aggregating the current sequence and the semantically nearest historical sequence representations through the attention mechanism. SmartUDI can be seamlessly deployed on cloud infrastructures of IoT device vendors and edge nodes, enabling the delivery of personalized device service recommendations to users. Comprehensive experiments on four real-world datasets show that SmartUDI consistently outperforms the state-of-the-art baselines with more accurate and highly interpretable results." https://doi.org/10.1145/36109062023JXJingyu Xiao et al.Context-Aware ComputingSmart Home Interaction DesignUbiComp
Generalization and Personalization of Mobile Sensing-Based Mood Inference Models: An Analysis of College Students in Eight CountriesMood inference with mobile sensing data has been studied in ubicomp literature over the last decade. This inference enables context-aware and personalized user experiences in general mobile apps and valuable feedback and interventions in mobile health apps. However, even though model generalization issues have been highlighted in many studies, the focus has always been on improving the accuracies of models using different sensing modalities and machine learning techniques, with datasets collected in homogeneous populations. In contrast, less attention has been given to studying the performance of mood inference models to assess whether models generalize to new countries. In this study, we collected a mobile sensing dataset with 329K self-reports from 678 participants in eight countries (China, Denmark, India, Italy, Mexico, Mongolia, Paraguay, UK) to assess the effect of geographical diversity on mood inference models. We define and evaluate country-specific (trained and tested within a country), continent-specific (trained and tested within a continent), country-agnostic (tested on a country not seen on training data), and multi-country (trained and tested with multiple countries) approaches trained on sensor data for two mood inference tasks with population-level (non-personalized) and hybrid (partially personalized) models. We show that partially personalized country-specific models perform the best yielding area under the receiver operating characteristic curve (AUROC) scores of the range 0.78--0.98 for two-class (negative vs. positive valence) and 0.76--0.94 for three-class (negative vs. neutral vs. positive valence) inference. Further, with the country-agnostic approach, we show that models do not perform well compared to country-specific settings, even when models are partially personalized. We also show that continent-specific models outperform multi-country models in the case of Europe. Overall, we uncover generalization issues of mood inference models to new countries and how the geographical similarity of countries might impact mood inference. https://dl.acm.org/doi/10.1145/35694832023LMLakmal Meegahapola et al.Mental Health Apps & Online Support CommunitiesContext-Aware ComputingUbiComp
SyncLabeling: A Synchronized Audio Segmentation Interface for Mobile DevicesManual audio segmentation is a time-consuming process, especially when there is more than one sound playing simultaneously that needs to be segmented and annotated (e.g., target and background sounds). In conventional audio annotation interfaces, users need to repeatedly pause and replay the audio to complete an overlap segmentation task, which is very inefficient. In this paper, we propose “SyncLabeling,” a synchronized audio segmentation interface for smartphones that allows users to segment and annotate two overlapping sounds in a single audio stream at a time using a game-like labeling interface on mobile devices. We conducted a user study to compare the proposed SyncLabeling interface with a conventional audio annotation interface on four types of audio segmentation tasks. The results showed that the proposed interface is much more efficient than the conventional interface (2.4× faster) under comparable annotation accuracy in most tasks. In addition, more than half of the participants enjoyed using the proposed SyncLabeling interface and showed willingness to use it.2023YTYi Tang et al.Gamification DesignMobileHCI
ParaGlassMenu: Towards Social-Friendly Subtle Interactions in ConversationsInteractions with digital devices during social settings can reduce social engagement and interrupt conversations. To overcome these drawbacks, we designed ParaGlassMenu, a semi-transparent circular menu that can be displayed around a conversation partner's face on Optical See-Through Head-Mounted Display (OHMD) and interacted subtly using a ring mouse. We evaluated ParaGlassMenu with several alternative approaches (Smartphone, Voice assistant, and Linear OHMD menus) by manipulating Internet-of-Things (IoT) devices in a simulated conversation setting with a digital partner. Results indicated that the ParaGlassMenu offered the best overall performance in balancing social engagement and digital interaction needs in conversations. To validate these findings, we conducted a second study in a realistic conversation scenario involving commodity IoT devices. Results confirmed the utility and social acceptance of the ParaGlassMenu. Based on the results, we discuss implications for designing attention-maintaining subtle interaction techniques on OHMDs.2023RCRunze Cai et al.National University of SingaporeAR Navigation & Context AwarenessMixed Reality WorkspacesContext-Aware ComputingCHI
Task Execution Quality Maximization for Mobile Crowdsourcing in Geo-Social NetworksWith the rapid development of smart devices and high-quality wireless technologies, mobile crowdsourcing (MCS) has been drawing increasing attention with its great potential in collaboratively completing complicated tasks on a large scale. A key issue toward successful MCS is participant recruitment, where a MCS platform directly recruits suitable crowd participants to execute outsourced tasks by physically traveling to specified locations. Recently, a novel recruitment strategy, namely Word-of-Mouth(WoM)-based MCS, has emerged to effectively improve recruitment effectiveness, by fully exploring users' mobility traces and social relationships on geo-social networks. Against this background, we study in this paper a novel problem, namely Expected Task Execution Quality Maximization (ETEQM) for MCS in geo-social networks, which strives to search a subset of seed users to maximize the expected task execution quality of all recruited participants, under a given incentive budget. To characterize the MCS task propagation process over geo-social networks, we first adopt a propagation tree structure to model the autonomous recruitment between the referrers and the referrals. Based on the model, we then formalize the task execution quality and devise a novel incentive mechanism by harnessing the business strategy of multi-level marketing. We formulate our ETEQM problem as a combinatorial optimization problem, and analyze its NP hardness and high-dimensional characteristics. Based on a cooperative co-evolution framework, we proposed a divide-and-conquer problem-solving approach named ETEQM-CC. We conduct extensive simulation experiments and a case study, verifying the effectiveness of our proposed approach.2021LWLiang Wang et al.Crowds and CollaborationCSCW
Get a Grip: Evaluating Grip Gestures for VR Input using a Lightweight PenThe use of Virtual Reality (VR) in applications such as data analysis, artistic creation, and clinical settings requires high precision input. However, the current design of handheld controllers, where wrist rotation is the primary input approach, does not exploit the human fingers' capability for dexterous movements for high precision pointing and selection. To address this issue, we investigated the characteristics and potential of using a pen as a VR input device. We conducted two studies. The first examined which pen grip allowed the largest range of motion---we found a tripod grip at the rear end of the shaft met this criterion. The second study investigated target selection via 'poking' and ray-casting, where we found the pen grip outperformed the traditional wrist-based input in both cases. Finally, we demonstrate potential applications enabled by VR pen input and grip postures.2020NLNianlong Li et al.Institute of Software, Chinese Academy of Sciences & University of Chinese Academy of SciencesFull-Body Interaction & Embodied InputSocial & Collaborative VRCHI