SmartPoser: Arm Pose Estimation With a Smartphone and Smartwatch Using UWB and IMU Data

Document Title

SmartPoser: Arm Pose Estimation with a Smartphone and Smartwatch Using UWB and IMU Data

Document Information

• Field of Study: Wearable Devices and Human Motion Recognition
• Keywords: Smartwatch, Sensing, Gesture Recognition, Body Posture, Mobile Devices, Interaction Technology

Research Background and Problem

• Identified Problems or Challenges:

  • Current smartwatches on the market have very limited understanding of users' arm posture information.
  • Existing arm posture tracking systems typically require external devices (e.g., cameras), which raise privacy concerns or are restricted to fixed locations. Additionally, solutions based on multiple IMU (Inertial Measurement Unit) sensors, though effective, are challenging to popularize among consumers.
  • Arm motion tracking is of significant value for applications such as fitness, rehabilitation training, augmented reality interaction, and context-aware assistants, but achieving a more convenient and mobile solution remains a challenge.

• Importance of the Problem:

  • Real-time understanding of users' arm posture can enhance capabilities in fields such as health management, motion recognition, and gaming interaction.
  • The hardware and software capabilities of smartphones and smartwatches, which are already owned by a large number of users, have not been fully utilized, offering a cost-effective and universal solution to the problem.

• Research Motivation and Related Work:

  • The authors reviewed the potential of fusing UWB (Ultra-Wideband) ranging and IMU sensor data for posture estimation.
  • Previous studies focused on distributed IMUs or solutions requiring additional installed equipment, which fail to achieve portability or real-time performance.
  • This study aims to address system drift and stability issues while avoiding the need for users to invest in additional complex hardware.

Solution

• Proposed Method or Solution:

  • The authors propose "SmartPoser," a system that achieves real-time 3D arm posture tracking using only off-the-shelf devices, namely a smartwatch and a smartphone.
  • The core innovation lies in fusing UWB absolute ranging data (less drift) with IMU data (providing orientation and acceleration information) to optimize posture estimation.

• Innovations:

  • The system is entirely based on off-the-shelf devices, requiring no additional hardware costs for users.
  • It is the first to introduce UWB data into wearable systems for posture tracking, leveraging UWB's absolute ranging capabilities to overcome drift issues in inertial sensor data.
  • A bidirectional LSTM (Long Short-Term Memory) network processes the data to improve tracking accuracy and reduce noise and errors in complex movements.

• Implementation Steps and Key Techniques:

  1. Device Selection and Data Collection: The iPhone 13 Pro and Apple Watch Series 7 were used as prototype devices for the experiment, collecting data via UWB and IMU.
  2. UWB Correction:
     • A dual-layer bidirectional LSTM network was designed to correct UWB ranging data, reducing linear drift and the impact of obstructions.
  3. Posture Estimation:
     • Based on the corrected UWB data and IMU sensor data, another dual-layer bidirectional LSTM was designed to predict 3D joint positions (including shoulder, elbow, and wrist).
  4. User Experiments and System Evaluation:
     • The system's performance was validated through real-world activities and a comprehensive dataset of static postures.
  5. Real-Time Output and Portability:
     • The method achieves real-time inference on a smartphone with a processing time of only 1ms, enabling predictions at a speed of 25 FPS.

Research Outcomes

• Specific Results:

  • SmartPoser achieved a median positioning error of 11.0 cm for wrist and elbow positions, comparable to many systems requiring professional equipment.
  • The system does not require users to provide training data, supporting a "plug-and-play" application model.
  • Designed for high portability, the system can operate on users' existing smart hardware devices.

• Comparison with Existing Solutions and Advantages:

  • Compared to existing solutions such as ArmTrak and IMUPoser, SmartPoser maintains equivalent or higher accuracy while supporting user mobility.
  • The combination of UWB and IMU data significantly reduces drift and instability issues associated with IMU-only systems, with error improvements ranging from 20% to 26%.

• Experimental or Evaluation Results:

  • In user experiments, SmartPoser demonstrated stable performance in tracking both dynamic daily activities and more challenging limb movements.
  • The model's accuracy was validated using joint data from Azure Kinect as ground truth.
  • UWB correction in the experiments significantly reduced the 35 cm-level errors caused by human body occlusion, ultimately controlling accuracy within 10 cm.

• Limitations and Future Directions:

  • Limitations: The current system is only applicable when the phone is placed in the front pocket of trousers; changes in device position require recalibration. Additionally, female users may face limitations due to the lack of functional pockets in clothing.
  • Future Improvements:
    1. Support for more phone carrying methods (e.g., backpacks or handheld).
    2. Simplification of calibration steps to a single gesture.
    3. Expansion to other UWB-enabled wearable devices, such as earbuds or smart glasses.
    4. Integration of enhanced device position detection modes to further improve user experience.

Conclusion

The SmartPoser project demonstrates the feasibility of using commercial smart devices for arm posture tracking. By fusing UWB and IMU data, the system achieves high-accuracy real-time tracking without increasing users' hardware costs. This opens up new possibilities for building smarter applications in fields such as fitness, rehabilitation, gaming, and context-aware assistants, offering significant research and practical value.