A Model Predictive Control Approach for Reach Redirection in Virtual Reality

Title of the Paper

A Model Predictive Control Approach for Reach Redirection in Virtual Reality

Paper Information

• Research Domain: Human-computer interaction and control models in virtual reality
• Keywords: Virtual reality, haptic redirection, model predictive control, optimization, motion control, hand trajectory, perception and feedback

Research Background and Problems

• What issues or challenges did the authors identify?

  • Current hand redirection algorithms often rely on predefined spatial interpolation methods, making it difficult to adapt to dynamic environments or individual differences among users.
  • Existing hand redirection methods typically do not account for users' perception and motion control processes, often employing "black-box" approaches to achieve hand offset.
  • There is a lack of research on how spatial distortion affects the underlying sensorimotor control models of user movement.
  • Balancing the trade-off between user experience and efficiency in manually adjusted redirection methods remains challenging.

• Why is this problem important?

  • In virtual reality, accurately simulating users' hand dynamics and motion control perception is crucial for enhancing the immersion and interaction experience with virtual objects.
  • Incorporating users' subjective experiences, motion comfort, and task completion efficiency into hand redirection algorithms can improve the robustness and generalizability of these methods.

• Motivation and Related Work

  • Inspired by human sensorimotor control models (e.g., Minimum Jerk model and feedback control principles), the authors aim to introduce dynamic system-based models into redirection strategies.
  • Model Predictive Control (MPC) enables real-time optimization of the mapping relationship between users' virtual and physical environments.
  • While some studies have applied MPC to virtual gait planning and haptic guidance, its application in hand redirection remains largely unexplored.

Solution

• What methods or solutions did the authors propose?

  1. Developed a feedback control dynamic system based on the Minimum Jerk (MJ) model to simulate hand trajectories influenced by redirection during user movement.
  2. Proposed a hand redirection framework based on Model Predictive Control (MPC), which generates optimal spatial distortions in real-time to meet redirection goals and sensory comfort constraints.
  3. Constructed two optimization objectives:
     • Endpoint-based Redirection: guiding the user's physical hand to reach specific spatial points.
     • Path-based Redirection: directing the user's physical hand to move along specific paths.

• What are the innovative aspects of this solution?

  • Modeled users' infrared hand offsets as a form of "sensory bias" in visual feedback estimation, capturing the sensorimotor control process.
  • Implemented a dynamically adjustable distortion generation method using MPC, surpassing linear interpolation strategies.
  • Incorporated cost functions for user experience (e.g., smoothness and acceptability) into the redirection objectives, directly balancing experience and efficiency.

• What are the implementation steps and key technologies used?

  1. Built an MJ feedback model for users' hand movements and simulated visual redirection through sensory bias.
  2. Designed a dynamic model in a multidimensional state space, including position, velocity, acceleration, and control inputs (infrared offsets).
  3. Defined optimization objective functions based on real-time goals (e.g., reaching a point or following a path).
  4. Transformed MPC into a discrete optimization problem in virtual reality experiments to generate real-time spatial distortion commands.
  5. Implemented the system in Unity and evaluated it using wearable devices (e.g., HTC Vive Pro Eye).

Research Outcomes

• What specific results were achieved?

  1. Preliminary simulation validation showed that the model could predict users' hand trajectories under redirection conditions, with errors within perceptual thresholds.
  2. In experiments, the endpoint redirection method based on MPC (MPC-E) demonstrated performance comparable to traditional methods (Haptic Retargeting, HR).
  3. For path redirection, the MPC strategy with higher weight learning (MPC-P0.1) performed better in terms of smoothness and accuracy but did not fully outperform the baseline method (Thin Plate Spline, TPS).

• What advantages does it have compared to existing solutions?

  • Dynamically adapts to changes in users' environments (e.g., obstacle avoidance or target adjustments).
  • Supports customizable optimization objectives, integrating users' subjective perceptions and task requirements.
  • Provides a more general framework that can be extended to other virtual reality interaction tasks.

• What were the experimental or evaluation results?

  • In two virtual reality interaction tasks involving 8 participants, the authors quantitatively compared the performance of MPC and traditional methods:
    • Endpoint Study: The endpoint error and redirection perceptibility of MPC-E were not significantly different from HR.
    • Path Study: MPC-P performed slightly worse than TPS in low-curvature scenarios but demonstrated better flexibility in high-curvature scenarios.
  • User reports indicated that MPC in path redirection had a higher learning cost, while TPS distortions were smoother and more intuitive.

• Limitations and Future Directions

  1. Experimental Design Limitations:
     • The range of offsets and configurations in the current experiments was limited, requiring further expansion of experimental conditions for comprehensive strategy evaluation.
     • Additional analysis is needed for scenarios balancing speed and accuracy trade-offs.
  2. Model Limitations:
     • The simplicity of the Minimum Jerk model failed to capture the complexity of multi-sensory integration.
     • Model performance needs to account for individual differences and dynamic task requirements in greater detail.
  3. Algorithm Optimization:
     • The computational cost and optimization problem-solving time of MPC remain significant limitations.
     • Further tuning of cost function weights is necessary to enhance user experience and system stability.
  4. Future Work Directions:
     • Improve the complexity of perception and motion models, such as incorporating multi-sensory channels.
     • Explore a wider variety of redirection tasks, optimizing for both visual and haptic feedback.