Automatic Tuning of Haptic Motion Effects to Evoke Specific Feelings in Multisensory Content

Research Background and Issues

• Problems or Challenges Identified by the Authors:
  With the development of multimodal media (e.g., 4D movies, VR games), haptic motion effects are widely applied to enhance content immersion. However, designing these effects requires addressing two core tasks: 1) generating motion effects consistent with video content; 2) fine-tuning these effects to evoke specific sensory or emotional experiences in users. Current research primarily focuses on automatically generating initial draft effects (e.g., motion generation based on camera trajectories) but lacks exploration of the subsequent fine-tuning process.

• Importance of the Problem:
  The purpose of fine-tuning haptic motion effects is to enhance the quality of user experience and convey the designer's intended user perceptions. If these sensory and emotional attributes are not effectively evoked, the user's multimodal experience may be significantly limited, preventing deep integration with video content.

• Research Motivation and Related Work:
  Although previous studies have explored the role of haptic stimuli in perception and emotion (e.g., haptic smoothness and emotional tension), they have largely neglected the relationship with video content. Furthermore, there is a lack of systematic automated generation methods that encompass user perception adjustments.

Solution

• Proposed Method or Solution:
  An automated method for adjusting haptic motion effects is proposed, including the following key steps: 1) adjusting motion effects using three motion modulation techniques, including low-frequency fluctuation enhancement, high-frequency component amplitude adjustment, and motion peak sampling interpolation; 2) designing user perception prediction models based on regression modeling, targeting the perception of motion and audiovisual effects; 3) mapping target perceptions to motion modulation parameters to achieve predefined sensory or emotional goals.

• Innovative Aspects of the Solution:

  1. For the first time, the influence of audiovisual effects on user perception is integrated, ensuring motion effect adjustments take audiovisual content into account.
  2. Three motion modulation methods are proposed, which can adjust user emotions and perceptions without compromising original visual consistency.
  3. A computational model is developed to automatically generate adjusted haptic motion effects through a two-step process (reverse derivation of motion parameters from target perceptions).

• Implementation Steps and Key Techniques:

  1. Perception Modeling: Through experimental research, regression models are established to map modulation parameters to user perceptions (e.g., "smooth-rough"); additionally, prediction models for perception changes under combined video and motion effects are developed.
  2. Target Mapping: Input target perceptions and video perception values to generate separate motion target perception values, and reverse-calculate corresponding modulation parameters using regression models.
  3. Modulation Implementation: Apply modulation techniques to generate motion effects that meet target perceptions.

Research Outcomes

• Specific Achievements:
  The proposed motion modulation methods and regression models were validated across two dimensions (perception and emotion). Experiments demonstrated that perceptions such as "smooth-rough" and emotions like "fear" could be accurately evoked through modulation methods, while effects on perceptions like "depth" were relatively limited.

• Advantages Over Existing Solutions:

  1. Simultaneously considers the comprehensive impact of audiovisual and motion effects on user experience, rather than focusing on single motion or perception effects.
  2. Provides an automated adjustment method that can be directly applied in production workflows, significantly reducing the time cost of iterative design adjustments.

• Experimental or Evaluation Results:

  1. Modulation effects on "smooth-rough" and "fear" emotions were most significant, with average Pearson correlation coefficients of 0.96 and 0.89, respectively, compared to target values.
  2. User experiments verified that the modulated motion effects closely aligned with designers' intended target perceptions, particularly in 4D movie scenarios.
  3. Feedback from designers indicated that the method significantly saved time, especially for POV (first-person perspective) video applications.

• Limitations and Future Directions:

  1. Certain user perceptions (e.g., "depth") are more strongly influenced by video content, limiting the current method's ability to effectively evoke target perceptions.
  2. Individual differences in perception may lead to variability in results. Additionally, applications in third-person perspective videos and complex narrative scenarios require further exploration.
  3. Future research will focus on developing more personalized models and exploring supplementary effects of other modalities (e.g., vibration and thermal sensations) on user perceptions.

Conclusion: This study significantly advances the frontier of automated haptic content generation, providing critical references and technical support for multimodal media design. Future efforts will prioritize model scalability and the integration of more diverse sensory fusion technologies.