TelePulse: Enhancing the Teleoperation Experience through Biomechanical Simulation-Based Electrical Muscle Stimulation in Virtual Reality

Research Background and Issues

• What problems or challenges did the authors identify?

  1. Current teleoperation technologies face insufficient interaction between users and remote robots, particularly in achieving force feedback within virtual reality (VR).
  2. Ground-based mechanical devices provide reliable force feedback but are bulky, expensive, and fixed in location, limiting user mobility and flexibility.
  3. Wearable devices (e.g., tactile feedback via vibration motors and electrical stimulation) offer greater flexibility but fail to deliver realistic three-dimensional force feedback.
  4. Electrical muscle stimulation (EMS), despite its lightweight and low-power advantages, struggles to accurately simulate multi-muscle coordination in complex dynamic environments.

• Why is this issue important?

  1. Teleoperation technology is widely applied in critical fields such as surgical operations, space exploration, and disaster rescue. Enhancing user experience and task efficiency is crucial for these scenarios.
  2. Providing more natural and accurate tactile feedback can reduce users' learning burden, minimize operational errors, and improve task success rates.

• Research Motivation and Related Work

  1. Existing studies primarily focus on ground-based force feedback devices and wearable vibration feedback devices, which fail to simultaneously offer high-quality force feedback and user freedom.
  2. Advances in science and technology have made it possible to address these issues by combining biomechanical simulation with EMS.

Solution

• What methods or solutions did the authors propose? TelePulse is a novel teleoperation system that integrates biomechanical simulation and electrical muscle stimulation (EMS) technology to provide users with realistic tactile feedback.

• Innovative aspects of the solution:

  1. Dynamic generation of joint torque estimates based on user posture and force data through biomechanical simulation, guiding EMS to deliver personalized muscle stimulation.
  2. Integration of biomechanical simulation with real-time VR environments to enhance the immersive user experience in teleoperation.
  3. Utilization of lightweight EMS devices to achieve a balance between high performance and portability.

• Implementation steps:

  1. Motion capture: Using Meta Quest 3 to record users' real-time motion postures.
  2. Biomechanical simulation: Calibrating and constructing virtual skeletal models based on users' geometric characteristics (e.g., height, weight, joint length).
  3. Joint force calculation: Collecting real-time force data via force-torque sensors and calculating target torque using two inverse dynamics models.
  4. Personalized EMS stimulation: Dynamically adjusting stimulation intensity and location based on the muscle force-intensity relationship curve.
  5. Real-time communication: Transmitting calculated results via Wi-Fi to EMS hardware and providing real-time muscle stimulation to generate tactile feedback.

• What key technologies were used?

  1. The biomechanical simulation platform OpenSim for modeling users' muscle and joint dynamics.
  2. Quaternion-based coordinate system transformation to ensure consistent force feedback direction between the remote machine and the user.
  3. Dynamic adjustment of EMS devices using digital potentiometers for precise signal control.

Research Outcomes

• What specific results were achieved?

  1. The TelePulse system significantly outperformed traditional EMS methods without biomechanical simulation (NonBioSim) in muscle activation accuracy.
  2. Users demonstrated significantly higher task completion accuracy and presence in experimental tasks.
  3. In continuous force tasks (e.g., polishing tasks), the mean absolute error (MAE) was reduced by 22%, while accuracy in impact force tasks (e.g., drilling tasks) improved by 30%.

• Advantages compared to existing solutions:

  1. Compared to devices offering only vibration tactile feedback, TelePulse provides richer force feedback information, including force direction and intensity.
  2. Compared to traditional linear proportional EMS stimulation schemes, TelePulse's biomechanical simulation avoids unnecessary muscle stimulation, enhancing realism and comfort during operation.

• Experimental or evaluation results:

  1. In the E1 (force-guided task) experiment, EMS signals generated by TelePulse showed significantly higher similarity to users' electromyography (EMG) data compared to non-simulation methods.
  2. In the E2 experiment, the TelePulse system improved users' performance in virtual tasks and demonstrated consistently high precision in both continuous force and impact force tasks.

• Limitations and future directions:

  1. Human muscle modeling is complex; currently, TelePulse is limited to simulating six arm muscles. Future work could expand to more complex systems, including shoulder and hand muscles.
  2. System personalization requires lengthy calibration (approximately 1 hour). Future improvements in computational models and optimization algorithms could reduce calibration time.
  3. Mild stinging sensations from EMS persist; exploring alternative technologies such as magnetic muscle stimulation may enhance user experience.
  4. Further validation of performance comparisons with other wearable devices (e.g., vibration motors and skin stretch actuators) is needed.

Through practical applications and in-depth experiments, TelePulse demonstrates its potential in providing user feedback stimulation for complex teleoperation tasks. It is foreseeable that TelePulse will have extensive applications in robotic control, immersive virtual reality, and industrial operation scenarios.