Can we crowdsource Tacton similarity perception and metaphor ratings?

Title of the Paper

Can we Crowdsource Tacton Similarity Perception and Metaphor Ratings?

Paper Information

• Domain: Human-Computer Interaction (HCI), Haptic Design, and User Experience
• Keywords: Crowdsourcing, Haptic Perception, Tacton Design, User Studies, Haptics

Research Background and Questions

Background

Tactile icons (Tactons) are widely used in fields such as virtual reality (VR), notification systems, autonomous driving, and medical rehabilitation. The key to designing tactile icons lies in optimizing their discriminability in the perceptual space while effectively mapping them to the information of the target application.

With the proliferation of high-fidelity vibration devices in modern smartphones, haptic designers can leverage crowdsourcing platforms to quickly evaluate haptic patterns. However, current research has primarily focused on laboratory-based design, with limited studies exploring the feasibility of achieving consistent evaluations through crowdsourcing methods.

Research Questions

1. Can crowdsourcing platforms achieve results consistent with laboratory settings in evaluating the similarity perception of tactile icons (Tactons)?
2. Can crowdsourcing provide consistent results for high-level cognitive tasks, such as evaluating the match between haptics and metaphors?
3. Are there background factors that influence haptic perception and metaphor ratings?

Significance

By validating the feasibility of crowdsourcing, designers can obtain user feedback more quickly and cost-effectively, significantly advancing the development of haptic design across various application domains.

Solution

Methods and Core Steps

The authors designed and conducted two user experiments to investigate the evaluation of tactile icon similarity and metaphor matching in both laboratory and crowdsourcing environments:

Experiment 1: Parametric Evaluation of Tacton Similarity

• Objective: To validate whether user perception of Tactons designed with systematic parameter control can be evaluated in a crowdsourcing environment.
• Design:
  • Constructed 12 tactile icons based on variations in three parameters (carrier frequency, duration, and modulation frequency).
  • 20 laboratory participants and 20 crowdsourced participants from Amazon MTurk completed similarity ratings (66 pairwise combinations).
  • Data analysis utilized Spearman correlation coefficients and non-metric multidimensional scaling (nMDS).

Experiment 2: Evaluation of Complex Tactons Based on Metaphors

• Objective: To test the consistency of similarity perception and metaphor matching for complex haptic icons designed based on metaphors.
• Design:
  • Selected 14 complex tactile icons corresponding to 7 metaphors from an existing icon library (VibViz).
  • Participants first completed a similarity rating task, followed by an evaluation of the match between icons and metaphors.
  • Equivalence testing was used to assess the consistency of metaphor matching.

Innovations

1. First validation of the correspondence between laboratory and crowdsourcing evaluations for tactile icons designed with systematic parameters and complex metaphors.
2. Provided a scalable method for crowdsourced haptics research, exploring a broader design parameter space enabled by high-fidelity haptic feedback on smartphones.

Technical Details

• Technology: Used the iPhone's "Taptic Engine" vibration sensor, debugging Apple Haptic and Audio Pattern (AHAP) files to ensure consistent haptic feedback across laboratory and crowdsourcing platforms.
• Device Calibration: Accelerometers measured vibrations, and parameters were adjusted to ensure consistency with target patterns.

Research Findings

Finding 1: Similarity Perception Evaluation

• Experiment 1

  • Statistical analysis showed a strong correlation between laboratory and crowdsourced similarity evaluations (𝜌=0.93, p<0.001).
  • The perceptual spaces in both environments aligned in dimensions (modulation frequency, duration, carrier frequency corresponded to perceptual dimensions).

• Experiment 2

  • Analysis of complex tactile icons revealed high consistency in similarity perception between laboratory and crowdsourced settings (𝜌=0.91, p<0.001).

Finding 2: Metaphor Ratings Evaluation

• Metaphor matching results indicated: 60% of ratings were statistically equivalent between laboratory and crowdsourced environments, 10% were significantly different, and 30% were inconclusive.
• Influencing Factors: Cultural background, environmental noise, and smartphone hardware may contribute to inconsistencies.

Advantages and Limitations

Advantages Over Existing Methods

• Crowdsourced settings demonstrated high consistency with laboratory data for certain tasks (especially perceptual tasks), saving time and cost for haptic designers.

Limitations

1. Lower consistency in metaphor evaluations: Crowdsourcing platforms may be significantly influenced by cognitive processes, background, and user context.
2. Hardware limitations: Consistency of vibration feedback across different smartphone models requires further validation.

Conclusion and Future Directions

Key Findings

• Crowdsourcing platforms can effectively evaluate perceptual tasks for tactile icons, but further control is needed for high-level cognitive tasks such as metaphor matching.
• The high-fidelity vibration feedback provided by iPhone devices lays a foundation for developing more efficient haptic design methods.

Future Directions

1. Further control of experimental variables: Investigate the impact of user background (e.g., gender, culture) and external factors (e.g., visual or auditory aids) on metaphor task consistency.
2. Expansion to other devices: Explore tactile icon design and evaluation for virtual reality and gaming controllers.
3. Development of a haptic similarity model: Use crowdsourcing to collect large-scale data and develop machine learning models to predict the perceptual similarity of tactile icons.