Touch&Fold: A Foldable Haptic Actuator for Rendering Touch in Mixed Reality

Title of the Paper

Touch&Fold: A Foldable Haptic Actuator for Rendering Touch in Mixed Reality

Paper Information

• Research Domain: Haptic rendering technology in Mixed Reality (MR)
• Keywords: haptics, wearable devices, mixed reality, haptic feedback, vibration, linear resonant actuator, self-contained device, tactile transmission, virtual and real-world interaction

Research Background and Problem Statement

• Research Problems and Challenges:

  • In Mixed Reality (MR), traditional haptic devices face limitations in preserving users' tactile perception of real-world objects while simultaneously rendering the tactile sensations of virtual objects.
  • Current devices, such as external vibrators, only provide tactile feedback on the fingernail and fail to deliver pressure sensations or realistic tactile experiences.
  • Existing solutions (e.g., patch-based actuator devices) attempt to minimize interference with users' tactile perception but still impact the fine tactile sensing of real-world objects.

• Significance: Mixed Reality applications often involve tasks such as tool operation, repair, and construction, requiring simultaneous tactile interaction with both real-world and virtual objects. Haptic feedback is crucial for enhancing task immersion and user experience.

• Motivation and Related Work:

  • Haptic feedback in Mixed Reality must provide realistic and precise tactile sensations without hindering users' ability to manipulate real-world objects.
  • The authors designed a miniature foldable haptic device that retracts from the user's fingertip when not in use, integrating haptic feedback into a more convenient form factor.

Solution

• Proposed Method or Solution:

  • A foldable fingernail-mounted haptic actuator for Mixed Reality, capable of providing tactile feedback when interacting with virtual objects and quickly retracting when handling real-world objects.

• Innovations:

  1. The first foldable device offering fingertip haptic feedback with rapid retraction capability.
  2. The device combines pressure and vibration to generate diverse tactile effects, including touch (pressure), high-frequency textures, and low-frequency textures.
  3. Self-contained design (including battery and electronics), wireless connectivity via Bluetooth communication.
  4. Transparent 3D-printed housing minimizes visual interference for users.

• Implementation Steps:

  1. Hardware Design:
     • Device dimensions: 24×24×41 mm, weight: 9.5 g, mounted on the fingernail.
     • Folding mechanism based on a gear-rack system, enabling the tactile cover to extend and contact the user's fingertip.
     • Integrated linear resonant actuator (LRA) for generating textured vibration feedback.
  2. Feedback Generation:
     • Touch sensation: The device applies pressure to the fingertip via a mechanical driving system.
     • Low-frequency textures (<25Hz): Generated through reciprocating motion of the actuator.
     • High-frequency textures (150-190Hz): Achieved by activating the LRA for vibration feedback.
  3. Software Support:
     • Utilizes Unity3D and HoloLens 2 for hand tracking and virtual interaction.
     • Defines virtual object touch detection zones, enabling the device to expand/retract in real-time based on interactions.

Research Outcomes

• Specific Results:

  • Successfully developed a fully self-contained haptic device capable of seamless tactile operation between real-world objects and virtual interfaces.
  • The device can render various tactile effects, including virtual surface sensations, textures, button clicks, etc.
  • Two user studies validated the device's tactile realism and operational convenience.

• Advantages Compared to Existing Solutions:

  • Compared to fingernail vibrators, the device significantly enhances tactile realism, especially for hard surfaces, buttons, and texture feedback.
  • The device imposes minimal interference during real-world object manipulation, improving user flexibility.

• Experimental or Evaluation Results:

  • First User Study:
    • Compared to fingernail vibrators, the device was rated as more realistic for tactile feedback across five virtual objects (hard surface, soft surface, button, low-frequency texture, high-frequency texture), except for soft surfaces.
    • All participants unanimously preferred the device's performance in texture and hard surface feedback.
  • Second User Study:
    • Participants completed complex repair tasks (e.g., using a screwdriver, checking brake fluid) without needing to remove the device.
    • Most participants reported that the device did not significantly affect their tactile perception of real-world objects.

• Limitations and Future Directions:

  • The device still covers part of the fingernail, potentially interfering with tactile perception on the sides of the fingertip.
  • Limited performance for soft object tactile feedback (force feedback is relatively rigid), requiring higher precision force sensors and controllers.
  • Mechanical motion delay of 92 ms needs compensation by expanding virtual collision zones.
  • Future directions include integrating additional haptic modules (e.g., thermal sensors, electrodes) and exploring efficient feedback mechanisms such as skin stretch or force illusions.