FIP: Endowing Robust Motion Capture on Daily Garment by Fusing Flex and Inertial SensorsWhat if our clothes could capture our body motion accurately? This paper introduces Flexible Inertial Poser (FIP), a novel motion-capturing system using daily garments with two elbow-attached flex sensors and four Inertial Measurement Units (IMUs). To address the inevitable sensor displacements in loose wearables which degrade joint tracking accuracy significantly, we identify the distinct characteristics of the flex and inertial sensor displacements and develop a Displacement Latent Diffusion Model and a Physics-informed Calibrator to compensate for sensor displacements based on such observations, resulting in a substantial improvement in motion capture accuracy. We also introduce a Pose Fusion Predictor to enhance multimodal sensor fusion. Extensive experiments demonstrate that our method achieves robust performance across varying body shapes and motions, significantly outperforming SOTA IMU approaches with a 19.5% improvement in angular error, a 26.4% improvement in elbow angular error, and a 30.1% improvement in positional error. FIP opens up opportunities for ubiquitous human-computer interactions and diverse interactive applications such as Metaverse, rehabilitation, and fitness analysis. Our project page can be seen at https://fangjw-0722.github.io/FIP.github.io/2025RZRuonan Zheng et al.Xiamen UniversityHaptic WearablesHuman Pose & Activity RecognitionCHI
CrowdBot: An Open-Environment Robot Management System for On-Campus ServicesWang 等人设计 CrowdBot 开放环境机器人管理系统，实现校园场景下机器人的自主导航与任务调度，为校园服务机器人的高效管理提供解决方案。2024YWYufei Wang et al.Domestic RobotsSocial Robot InteractionUbiComp
TouchEditor: Interaction Design and Evaluation of a Flexible Touchpad for Text Editing of Head-Mounted Displays in Speech-unfriendly EnvironmentsZhan 等人设计 TouchEditor 柔性触控板，解决语音不友好环境下头显显示器的文本编辑交互问题。2024LZLishuang Zhan et al.Head-Up Display (HUD) & Advanced Driver Assistance Systems (ADAS)Hand Gesture RecognitionVoice User Interface (VUI) DesignUbiComp
CrowdQ: Predicting the Queue State of Hospital Emergency Department Using Crowdsensing Mobility Data-Driven Models"Hospital Emergency Departments (EDs) are essential for providing emergency medical services, yet often overwhelmed due to increasing healthcare demand. Current methods for monitoring ED queue states, such as manual monitoring, video surveillance, and front-desk registration are inefficient, invasive, and delayed to provide real-time updates. To address these challenges, this paper proposes a novel framework, CrowdQ, which harnesses spatiotemporal crowdsensing data for real-time ED demand sensing, queue state modeling, and prediction. By utilizing vehicle trajectory and urban geographic environment data, CrowdQ can accurately estimate emergency visits from noisy traffic flows. Furthermore, it employs queueing theory to model the complex emergency service process with medical service data, effectively considering spatiotemporal dependencies and event context impact on ED queue states. Experiments conducted on large-scale crowdsensing urban traffic datasets and hospital information system datasets from Xiamen City demonstrate the framework's effectiveness. It achieves an F1 score of 0.93 in ED demand identification, effectively models the ED queue state of key hospitals, and reduces the error in queue state prediction by 18.5%-71.3% compared to baseline methods. CrowdQ, therefore, offers valuable alternatives for public emergency treatment information disclosure and maximized medical resource allocation." https://doi.org/10.1145/36108752023TSTieqi Shou et al.Content Moderation & Platform GovernancePublic Transit & Trip PlanningUbiComp
A Data-Driven Context-Aware Health Inference System for Children during School Closures"Many countries have implemented school closures due to the outbreak of the COVID-19 pandemic, which has inevitably affected children's physical and mental health. It is vital for parents to pay special attention to their children's health status during school closures. However, it is difficult for parents to recognize the changes in their children's health, especially without visible symptoms, such as psychosocial functioning in mental health. Moreover, healthcare resources and understanding of the health and societal impact of COVID-19 are quite limited during the pandemic. Against this background, we collected real-world datasets from 1,172 children in Hong Kong during four time periods under different pandemic and school closure conditions from September 2019 to January 2022. Based on these data, we first perform exploratory data analysis to explore the impact of school closures on six health indicators, including physical activity intensity, physical functioning, self-rated health, psychosocial functioning, resilience, and connectedness. We further study the correlation between children's contextual characteristics (i.e., demographics, socioeconomic status, electronic device usage patterns, financial satisfaction, academic performance, sleep pattern, exercise habits, and dietary patterns) and the six health indicators. Subsequently, a health inference system is designed and developed to infer children's health status based on their contextual features to derive the risk factors of the six health indicators. The evaluation and case studies on real-world datasets show that this health inference system can help parents and authorities better understand key factors correlated with children's health status during school closures. https://doi.org/10.1145/3580800"2023ZJZhihan Jiang et al.Cognitive Impairment & Neurodiversity (Autism, ADHD, Dyslexia)Mental Health Apps & Online Support CommunitiesUbiComp
vMirror: Enhancing the Interaction with Occluded or Distant Objects in VR with Virtual MirrorsInteracting with out of reach or occluded VR objects can be cumbersome. Although users can change their position and orientation, such as via teleporting, to help observe and select, doing so frequently may cause loss of spatial orientation or motion sickness. We present vMirror, an interactive widget leveraging reflection of mirrors to observe and select distant or occluded objects. We first designed interaction techniques for placing mirrors and interacting with objects through mirrors. We then conducted a formative study to explore a semi-automated mirror placement method with manual adjustments. Next, we conducted a target-selection experiment to measure the effect of the mirror's orientation on users' performance. Results showed that vMirror can be as efficient as direct target selection for most mirror orientations. We further compared vMirror with teleport technique in a virtual treasure hunt game and measured participants’ task performance and subjective experiences. Finally, we discuss vMirorr user experience and present future directions.2021NLNianlong Li et al.Institute of Software, Chinese Academy of Sciences, Institute of Software, Chinese Academy of SciencesSocial & Collaborative VRImmersion & Presence ResearchCHI
Sensock: 3D Foot Reconstruction with Flexible SensorsCapturing 3D foot models is important for applications such as manufacturing customized shoes and creating clubfoot orthotics. In this paper, we propose a novel prototype, Sensock, to offer a fully wearable solution for the task of 3D foot reconstruction. The prototype consists of four soft stretchable sensors, made from silk fibroin yarn. We identify four characteristic foot girths based on the existing knowledge of foot anatomy, and measure their lengths with the resistance value of the stretchable sensors. A learning-based model is trained offline and maps the foot girths to the corresponding 3D foot shapes. We compare our method with existing solutions using red–green–blue (RGB) or RGBD (RGB-depth) cameras, and show the advantages of our method in terms of both efficiency and accuracy. In the user experiment, we find that the relative error of Sensock is lower than 0.55%. It performs consistently across different trials and is considered comfortable and suitable for long-term wearing.2020HZHechuan Zhang et al.Xiamen UniversityFull-Body Interaction & Embodied InputCircuit Making & Hardware PrototypingCustomizable & Personalized ObjectsCHI