FingerX: Rendering Haptic Shape of Virtual Objects Augmented from Real Objects using Extendable and Withdrawable Supports on Fingers

Title of the Paper

FingerX: Rendering Haptic Shapes of Virtual Objects Augmented by Real Objects using Extendable and Withdrawable Supports on Fingers

Paper Information

• Research Domain: Haptic feedback and shape rendering technologies in the fields of Virtual Reality (VR) and Augmented Reality (AR)
• Keywords: Haptic feedback, shape rendering, wearable devices, virtual reality, augmented reality, hand interaction, multimodal interaction

Research Background and Problem Statement

• Problems or Challenges:
  1. Existing haptic feedback devices typically focus on virtual objects and fail to support seamless interaction with real objects.
  2. Current devices (e.g., vibration actuators and mechanical braking systems) restrict finger mobility, making it difficult to switch seamlessly between virtual and real objects.
  3. When rendering the shapes of virtual objects, the quality of feedback and flexibility of interaction remain insufficient.
  4. Users struggle to achieve precise haptic feedback for virtual objects while interacting naturally with real-world objects.
• Significance:
  1. As VR and AR technologies advance, users increasingly prefer interacting in mixed environments, where virtual objects coexist with real-world settings.
  2. Enhancing the realism of haptic feedback is crucial for improving the quality of interaction between virtual and real elements.
• Research Motivation:
  1. Strive to develop an interactive device capable of seamless switching between real and virtual objects.
  2. Create a device that can independently control the height and direction of feedback provided to fingertips in scenarios involving both virtual and real objects.

Solution

• Method or Solution: The authors propose a wearable haptic device called FingerX, which uses extendable and withdrawable supports mounted on users' fingers to achieve "dynamic rendering" of virtual shapes while enabling simultaneous interaction with both real and virtual objects.

• Innovations:

  1. By utilizing extendable and withdrawable support devices (Extenders and Rings), the design addresses the challenge of simultaneously touching real and virtual objects.
  2. The device independently controls the height and direction for each finger, enabling more realistic haptic feedback.
  3. The extendable mechanism avoids restricting the posterior movement of fingers, allowing for a more natural user experience.
  4. The switchable mode allows the device to provide haptic feedback while permitting direct finger contact with real objects.

• Implementation Steps and Key Technologies:

  1. Hardware Design:
     • Designed a five-level mechanical scissor lift support controlled by miniature motors to adjust height.
     • Added a circular rotation mechanism enabling automatic retraction of the support when necessary.
  2. Software Implementation:
     • Fingers are initially set to a state where all extenders are retracted.
     • The system tracks the distance between fingers and virtual objects, calculates the required extension height, and drives the device to render the shape.
     • Provides synchronized delay compensation for rotation and extension, including visual animations and sound notifications.
  3. Interaction Scenarios:
     • Offers three typical application scenarios: touching virtual objects, grasping virtual objects augmented by real objects, and holding purely virtual objects.
  4. User Experiments:
     • Shape recognition experiment: Evaluates users' ability to recognize shapes with and without visual feedback.
     • VR experience experiment: Compares FingerX with vibration feedback methods in interactions involving both virtual and real objects.

Research Outcomes

• Specific Results:

  1. Shape Recognition Experiment:
     • Average shape recognition rate without visual feedback was 61.71%; with visual feedback, the rate improved to 76.59%, showing stability across three different interaction scenarios.
  2. VR Experience Experiment:
     • Users supported by FingerX were able to clearly perceive the shapes of virtual objects while simultaneously interacting with real objects.
     • Compared to vibration feedback, FingerX significantly enhanced the "realism" and "distinguishability" of interactions involving virtual and real objects.

• Advantages:

  1. Compared to existing vibration devices, FingerX better simulates realistic haptics, making it ideal for mixed environments requiring precise haptic feedback.
  2. The support design significantly improves interaction with virtually augmented real objects (e.g., tables, bottles).
  3. Users can seamlessly switch between haptic perception of virtual and real objects without frequently removing the device.

• Limitations:

  1. The current device's haptic resolution is relatively low, rendering only limited shapes and struggling to accurately represent curved or highly complex shapes.
  2. Stability issues arise, particularly when interacting with curved surfaces (e.g., bottles), where the device tends to slip.
  3. Extension delays (up to approximately 6300ms) need further optimization to improve real-time performance.
  4. Ineffective in providing haptic feedback for sliding gestures, limiting its functionality to contact and grasp operations.

• Future Directions:

  1. Introduce higher-resolution haptic rendering technologies to enhance the device's capability in representing complex shapes (e.g., curved surfaces).
  2. Improve device stability by implementing more precise fingertip tracking systems and optimizing materials used in the hardware.
  3. Address issues related to device size and response time by exploring more efficient mechanical and motor designs.
  4. Apply FingerX to augmented reality (AR) scenarios to validate its applicability in various mixed reality environments.