Visuo-haptic Crossmodal Shape Perception Model for Shape-Changing Handheld Controllers Bridged by Inertial Tensor

Title of the Paper

Visuo-haptic Crossmodal Shape Perception Model for Shape-Changing Handheld Controllers Bridged by Inertial Tensor

Paper Information

• Subject Area: Human-Computer Interaction, Multisensory Shape Perception in Virtual Reality, and Handheld Controller Design
• Keywords: Handheld Controllers, Perception Model, Virtual Reality, Dynamic Haptics, Shape-Changing

Research Background and Problem

• Problems and Challenges:

  • Current handheld controllers in Virtual Reality (VR) systems typically have fixed shapes, leading to inconsistencies between visual and haptic perceptions of virtual objects, which may reduce user immersion.
  • Existing research on dynamic haptics is limited and lacks broadly applicable computational perception models for design purposes.

• Research Significance:

  • Dynamic haptics play a critical role in the perception of weight, shape, and other properties of handheld objects, yet the ability to match virtual and physical characteristics remains underdeveloped.
  • Providing a consistent visuo-haptic multisensory experience in VR is fundamental to achieving a highly immersive experience.

• Motivation and Related Work:

  • Some studies have begun exploring "shape-changing controllers" to enhance physical interaction in VR, but existing methods are limited in terms of model generalizability and perceptual consistency.
  • There is a need for more precise perception models to guide sensory matching, particularly dynamic haptic models based on mass distribution variables.

Proposed Solution

• Methodology and Core Ideas:

  • A visuo-haptic crossmodal perception model is proposed, based on the principal moments of inertia (MOI) and products of inertia (POI) derived from the inertial tensor.
  • Experiments explore the relationship between MOI and POI in dynamic haptic perception and object properties such as length and asymmetry, enabling consistent perception between virtual and physical objects.

• Innovations:

  • The first generalizable model is proposed to support consistent perceptual experiences between virtual objects and physical controllers.
  • MOI and POI are introduced as bridging variables linking visual and haptic perceptions.
  • An inverse model is employed to achieve object shape matching from visual to haptic modalities.

• Implementation Steps:

  1. Simulation Analysis:
     • Simulate mass distribution characteristics to verify that MOI and POI are key variables suitable for representing most handheld controller shapes.
  2. Human Perception Experiments:
     • Measure the threshold and sensitivity of MOI in length perception (Exp.1).
     • Measure the perception threshold of POI in mass asymmetry (Exp.2).
  3. Crossmodal Model Construction:
     • Conduct perception matching experiments to establish a crossmodal shape-matching model from haptic to visual modalities (Exp.3) and use its inverse form to achieve virtual-to-haptic object matching.
  4. Model Validation:
     • Validate the inverse model's improvement of user experience through evaluations in real-world scenarios (Exp.4).

Research Outcomes

• Specific Results:

  1. MOI and POI as Key Perceptual Variables:
     • MOI dominates length perception with a high sensitivity of 7% (Weber fraction).
     • POI significantly influences the perception of object asymmetry.
  2. Crossmodal Matching Model:
     • A high-precision perception model was established to map physical objects (MOI, POI) to virtual objects, demonstrating effectiveness in dynamic interactions.
  3. User Experience Testing of the Inverse Model:
     • In virtual scenarios, physical controllers matched to the visual information of virtual objects significantly enhanced immersion, consistency, and user enjoyment.

• Advantages:

  • Provides a generalizable computational model applicable to most 1D and some 2D shape-changing controller designs.
  • Offers more robust and broadly applicable potential compared to methods based on perceptual illusions.

• Experimental or Evaluation Results:

  • Human perception experiments demonstrated high sensitivity of MOI to length and POI to asymmetry perception.
  • User experiments (Exp.4) showed that controllers matched based on the model achieved the highest user perception experience score of 78.56 compared to unmatched controllers.

• Limitations and Future Directions:

  • The current model only covers objects with mass distribution in a semi-plane, limiting its application to fully 2D or 3D shape-changing controllers.
  • The interaction between MOI and POI in length perception has not been studied.
  • Gender differences (e.g., in perceptual performance) may also be a direction for further exploration.