DiscoBand: Multiview Depth-Sensing Smartwatch Strap for Hand, Arm and Environment Tracking

Document Title

DiscoBand: Multiview Depth-Sensing Smartwatch Strap for Hand, Body, and Environment Tracking

Document Information

• Subject Area: Smart Wearable Devices and Human-Computer Interaction
• Keywords: Smartwatch, Depth Sensing, Gesture Recognition, Body Posture, Mobile Devices, Interaction Technology

Research Background and Problem

• Problem or Challenge: Real-time tracking of hands, arms, and the environment is crucial in many human-computer interaction domains. However, existing tracking solutions often face the following issues:
  • Fixed Nature: Dependence on external tracking devices conflicts with the need for user mobility.
  • Privacy Concerns: High-resolution cameras are commonly used, raising privacy and security concerns.
  • Self-Occlusion of Hands: Gestures may result in data loss due to occlusion during imaging.
• Significance: Addressing these issues can not only enhance current technologies but also expand the practical applications of hand posture tracking, such as in virtual reality, gesture control, and healthcare.
• Research Motivation and Related Work:
  • The authors reviewed three main methods for hand tracking: solutions with instrumentation for both the environment and the user, those with instrumentation for the environment only, and those with instrumentation for the user only.
  • Additionally, the wrist is considered an advantageous position for instrumentation, as it is relatively close to the hand and can capture relevant data.

Solution

• Method or Solution: The research team proposed a wristband named "DiscoBand," which integrates 16 low-resolution, ultra-miniature depth sensors. Eight sensors are used to capture hand data from multiple angles, while the other eight track the user's body and environment.
• Innovations:
  • A multi-view sensor layout effectively reduces data loss caused by occlusion.
  • The use of low-resolution depth sensors enhances privacy protection while enabling a compact hardware design (≤1cm thickness).
  • Extended functionalities, such as estimating upper body posture, detecting handheld objects, and scanning the environment, are implemented.
• Implementation Steps and Key Technologies:
  • Hardware Design Optimization: Flexible PCB design with integrated VL53L5CX time-of-flight sensors, each providing 8×8-pixel depth images.
  • Data Processing and Software Support: Real-time synthesis of data from multiple sensors to construct a unified 3D point cloud, with specific machine learning models used for hand and body posture prediction.
  • Endpoint Calibration: Optimization of the model using user-specific body parameters (e.g., arm length, palm size) to adapt to different users.

Research Outcomes

• Specific Results:
  • DiscoBand achieves low-error gesture tracking and demonstrates high accuracy in hand and body posture tracking.
  • During testing, the mean per-joint position error (MPJPE) for hand posture was 11.7mm during single-use, while the joint position error for upper body posture was 5.88cm.
  • Additional applications explored include detecting handheld objects, environmental scanning, body scanning, and dual-hand activity tracking.
• Advantages:
  • Compared to existing solutions, DiscoBand offers better privacy protection along with a more compact and flexible design.
  • Its multi-view data capability provides stronger resistance to occlusion.
• Limitations and Future Directions:
  • Outdoor usage is affected by environmental light interference (e.g., infrared interference).
  • Changes in wearing position significantly impact tracking performance, and cross-user performance still requires improvement.
  • The large number of sensors may lead to high costs; future sensor generations could simplify the system by improving individual sensor performance.
  • Power consumption remains an issue, necessitating further exploration of intermittent sensor operation modes.
  • Future research could explore more data fusion techniques, such as integrating electrical impedance tomography (EIT) and surface electromyography (EMG), to enhance system robustness.

This study provides an innovative and highly promising solution for hand and body posture tracking while outlining future research directions and application scenarios.