UltraPoser: Pushing the Limits of IMU-based Full-Body Pose Estimation with Ultrasound Sensing on Consumer Wearables

DOI

Full-body motion capture using IMUs embedded in consumer wearables has the potential to enable convenient, on-the-go tracking with minimal instrumentation. However, the sparse placement of these devices on the body frame presents challenges such as limited body coverage, reduced motion feature diversity, and cumulative drift errors. This paper introduces UltraPoser, a multi-modal full-body motion capture system that integrates ultrasonic sensing with inertial measurements for improved fidelity, broader coverage, and increased reliability. UltraPoser leverages built-in microphones and speakers on commodity wearables, such as smartphones and smartwatches, to transmit and receive inaudible ultrasound signals, expanding the range of sensed body areas and providing drift-free acoustic multipath profiles. To implement UltraPoser, we systematically explore ultrasound signal designs to maximize feature quality and propose a graph-based physics-aware fusion architecture to integrate heterogeneous sensing modalities. We evaluate our approach using the UltraPoser Dataset, collected from 10 participants across diverse device placements and activity contexts. Compared to state-of-the-art IMU-only methods, UltraPoser achieves a 28.46% improvement in overall pose estimation accuracy and up to 67.28% error reduction for specific limbs without directly attached sensors.