HapticLens: Interactive Vibrotactile Haptic Generation from Spatially Localized Video Motion

Paper Title

HapticLens: Interactive Vibrotactile Haptic Generation from Spatially Localized Video Motion

Publication Info

• Topic area: Interactive haptic signal generation from video motion for single-actuator devices.
• Keywords: Haptics, video-to-haptics, vibrotactile feedback, motion processing, saliency estimation, single actuator, interactive design, user study, GPU acceleration, multimodal interaction.

Background and Problem

• Problem / challenge: Existing video-to-haptics methods depend on specific hardware (e.g., spatial haptic devices) or predefined motion characteristics, limiting their generalizability and accessibility for single-actuator devices like smartphones or VR controllers. Fully automated methods also reduce creative control.
• Significance: Expanding haptic content creation to arbitrary video content and single-actuator devices can lower barriers for designers, enhance user immersion, and broaden applications in gaming, VR, and mobile devices.
• Motivation and related work: Prior work has explored spatial haptics (e.g., motion chairs, vibrotactile arrays) and audio-to-haptics pipelines but lacks general solutions for arbitrary video content or interactive workflows. This paper builds on computer vision techniques like phase-based motion processing and saliency estimation to address these gaps.

Solution

• Proposed approach: HapticLens, an interactive method for generating vibrotactile haptic signals from user-selected video regions, using two vision algorithms (Phase-Based and Saliency-Based) and a GPU-accelerated implementation.
• Novelty:
  1. Interactive video-to-haptics workflow enabling region-based control for single-actuator devices.
  2. Integration of phase-based motion processing and spatiotemporal saliency estimation for dynamic feature extraction.
  3. Open-source, GPU-accelerated implementation with a graphical user interface for novice designers.
• Procedure and key techniques:
  • Users select a region of interest in a video.
  • Dynamic visual features (motion, saliency) are extracted using computer vision algorithms.
  • Features are converted into vibrotactile signals via amplitude and frequency modulation.
  • Users audition and refine the generated vibrations interactively.

Results

• Concrete findings:
  • Phase-Based algorithm showed higher sensitivity to motion dynamics and slightly higher ratings for overall signal quality (AVG=87.73) compared to Saliency-Based (AVG=82.56).
  • Both algorithms achieved high relevance ratings to video content (Phase-Based AVG=86.19; Saliency-Based AVG=82.19).
  • Design times per video averaged under 90 seconds, demonstrating efficiency.
• Advantage over baselines:
  • Supports arbitrary video content and single-actuator devices, unlike prior methods targeting spatial haptics or predefined motion types.
  • Offers interactive control, enabling creative flexibility absent in fully automated pipelines.
• Experiments / evaluation:
  • Technical assessments: Sensitivity to motion, robustness to noise and resolution, runtime performance.
  • User study with 22 participants: Evaluated usability, signal quality, and relevance across five diverse video clips.
  • Benchmarks: Processing times for vision algorithms and haptic generation were suitable for real-time interaction (<8ms for signal generation).
• Limitations and future work:
  • Limited evaluation across video genres; future work should test broader video datasets and longer clips.
  • Participant diversity focused on novices; further studies with expert designers are needed.
  • Generated signals are perceptually grounded but not physically accurate; future research could explore multimodal workflows combining visual and audio cues.

Summary

HapticLens introduces an interactive method for generating vibrotactile haptic signals from video motion, targeting single-actuator devices like smartphones and VR controllers. By leveraging phase-based motion processing and saliency estimation, it enables region-specific haptic design with high user satisfaction and efficiency. Technical evaluations and a user study demonstrate its feasibility, robustness, and accessibility for novice designers. Future directions include expanding video datasets, integrating multimodal cues, and exploring automatic region selection for enhanced usability. HapticLens broadens the scope of haptic content creation, offering practical tools for immersive applications in gaming, VR, and beyond.