Exploring the Remapping Impact of Spatial Head-hand Relations in Immersive Telesurgery

Research Background and Problem

• Identified Issues or Challenges:
  This study focuses on the remapping of spatial head-hand relationships caused by changes in head-mounted camera positions and robotic arm switching during immersive remote surgery. Such remapping leads to visual-proprioceptive conflicts, increasing cognitive load and operational errors. These issues affect users' spatial perception and performance, particularly in precise hand-eye coordination tasks.

• Importance of the Problem:
  Spatial perception and operational precision are critical during complex operations in remote surgery. Sensory conflicts caused by remapping can significantly reduce the interaction efficiency between surgical equipment and users, increasing safety risks. This study aims to identify these sensory conflicts and provide new insights for optimizing remote surgery system design.

• Research Motivation and Related Work:
  Related studies indicate that sensory conflicts (e.g., visual-vestibular or visual-proprioceptive conflicts) lead to reduced motion accuracy, cognitive fatigue, and motion discomfort, affecting user performance. However, systematic research on sensory conflicts caused by head-hand relationship remapping in immersive remote surgery remains absent. This study seeks to fill this knowledge gap and explore optimization methods.

Solution

• Proposed Methods or Solutions:
  The authors designed a virtual reality-driven simulated remote surgery system to test the effects of different head camera and robotic arm remapping conditions on perceptual bias, operational deviation, cognitive load, and task completion time. A total of 18 remapping scenarios were designed, covering different targets (head camera and robotic arm), attributes (offset and rotation), and parameter intensities (strong and weak).

• Innovative Aspects of the Solution:

  1. Systematically defined visual-proprioceptive conflicts caused by head-hand relationship remapping in immersive remote surgery and their physical impacts on behavior for the first time.
  2. Proposed a theoretical explanation of the perception-behavior mechanism, analyzing the formation and effects of conflicts based on multisensory integration theory.
  3. Conducted comparative analysis of remapping targets, attributes, and intensities, ranking the most influential scenarios to guide the optimization of surgical system design and usage.

• Implementation Steps and Key Technologies:

  1. Developed a simulated remote surgery environment using Blender and Unity, referencing instruments and environments used in real surgeries.
  2. Configured offset and rotation remapping parameters for head cameras and robotic arms, simulating various surgical tasks (e.g., navigation, positioning, cutting, and bimanual coordination) in VR.
  3. Designed quantitative and qualitative evaluation metrics, including spatial perception bias, trajectory deviation, NASA-TLX workload scores, and task completion time.
  4. Collected data through participant experiments and analyzed influencing mechanisms using feedback from interviews.

Research Findings

• Specific Results Achieved:

  1. Key Findings: Strong angular rotation of the head camera had the greatest impact on spatial perception bias, while strong offset of the robotic arm significantly affected physical burden and operational constraints.
  2. Scenario Ranking: Analyzed the effects of 18 remapping scenarios, identifying camera rotation (e.g., clockwise) and robotic arm forward-backward offset as the most significant factors.
  3. Multisensory Conflict Mechanism: Visual-proprioceptive conflicts primarily stem from users compensating for movement direction and adjusting sensory integration processes, increasing cognitive load and bias.

• Advantages Compared to Existing Solutions:

  1. Distinction Between Global and Local Effects: Systematically compared the global sensory integration impact of head cameras with the local action reference frame influence of robotic arms.
  2. Multidimensional Analysis: Simultaneously evaluated the effects of remapping on perception, behavior, and cognitive load, providing a more comprehensive understanding than previous studies.

• Experimental or Evaluation Results:
  Data analysis revealed that rotational remapping of the head camera had the broadest negative impact, while offset remapping of the robotic arm primarily affected physical coordination and workload. Weak parameter settings had less impact on sensory conflicts than strong settings but were still significantly higher than conditions without remapping.

• Limitations and Future Directions:

  1. Simulation System Limitations: Real surgeries involve more complex factors such as force feedback and dynamic instrument behavior, which were only partially simulated in this VR system.
  2. Participant Type: The experiment involved ordinary university students rather than professional surgeons, potentially underestimating the impact of expertise. Future research will target experts.
  3. Validation in Real Scenarios: Future work will directly test the findings in surgery-related scenarios.
  4. Research on Other Senses: Future studies will integrate more sensory factors such as haptics and vestibular systems to construct a more comprehensive optimization model.

Conclusion

This study systematically defined and explored the sensory conflicts and physical impacts caused by spatial head-hand relationship remapping in immersive remote surgery for the first time. It revealed the negative effects of these factors on spatial perception, action performance, and workload, and proposed effective optimization recommendations. The findings provide valuable references for improving remote surgery systems and designing surgical training methods, while also being applicable to related fields such as virtual reality and teleoperated robotics.