Squishy, Yet Satisfying: Exploring Deformable Shapes' Cross-Modal Correspondences with Colours and Emotions

Literature Title

Squishy, Yet Satisfying: Exploring Deformable Shapes’ Cross-Modal Correspondences with Colours and Emotions

Literature Information

• Subject Area: Human-Computer Interaction (HCI), exploring cross-modal correspondences between deformable shapes, colours, and emotions
• Keywords: deformable, shapes, colours, emotions, cross-modal correspondence, force, haptics, visuo-tactile, tactile-only

Research Background and Questions

• Research Background:
  • Deformable and shape-changing interfaces enrich the interaction methods of computational devices through physical interaction, but foundational design principles connecting these intrinsic shape attributes to human cognitive perception are lacking.
  • Cross-modal Correspondence (CC) phenomena reveal how the brain integrates multisensory information, extending from neuroscience and psychology to HCI, guiding designers to combine emotional experiences across visual and tactile modalities.
• Main Questions:
  • How do users associate deformable shapes with varying stiffness and angles to colours and emotions?
  • Does the visual context influence users' perception of these shapes?
• Research Motivation and Questions:
  • Propose design principles for measuring user perception through CC: understanding the design drivers derived from visual and tactile interactions with colours and emotions.
  • Research questions:
    • RQ1: How do people interact with deformable shapes of varying stiffness, and how do they associate these shapes with emotions and colours?
    • RQ2: Do these emotional and colour associations change when visual observation is not possible?

Solution

• Research Approach:
  • Conduct user experiments to measure the impact of shape, stiffness, and visibility on colour and emotion associations under visuo-tactile and tactile-only conditions. Physical shapes simulating user pressing interactions were used, including soft, medium, and hard stiffness levels.
  • Participants: 52 participants interacted with 24 physical shapes.
  • Emotion Measurement Tool: The Pleasure-Arousal-Dominance (PAD) model was used.
  • Innovations:
    1. First analysis of the impact of stiffness and angular properties on consistent associations under visuo-tactile and tactile-only conditions.
    2. Proposed interface design guidelines addressing how different sensory stimuli map to human cognition and emotions.

Experiment Implementation and Key Techniques

1. Experimental Materials: Developed four shapes (Curve, Sinuous, Emerge, and Porous) with three levels of stiffness: Soft, Medium, and Hard.
2. Testing Tasks:
   • Colour Task: Participants selected colours through visual and tactile interaction.
   • Emotion Task: Participants reported levels of pleasure, arousal, and dominance based on tactile experiences.
3. Technical Methods:
   • 3D-printed molds and silicone casting were used to create experimental materials.
   • Pressure sensors recorded user interaction forces during the experiment.
   • Data analysis included ANOVA and qualitative analysis methods.

Research Findings

• Key Findings:

  1. Colour Associations: Soft shapes were associated with cool colours (green, blue) and high brightness; hard shapes were associated with warm colours (red) and low brightness.
  2. Emotion Associations:
     • Soft, rounded shapes elicited higher pleasure, while sharp, hard shapes induced displeasure.
     • Rounded shapes conveyed a sense of high control, whereas sharp shapes increased a sense of instability.
     • Sharp shapes evoked higher arousal, while rounded shapes were more calming.
  3. Visual Conditions: Users' colour and emotion associations were consistent regardless of whether they could see the shapes.
  4. Force Analysis: Users tended to press harder when they could not see the shapes.

• Design Guidelines:

  • Appropriately use soft shapes and colour combinations to encourage more intuitive interactions, such as designing with cool tones and rounded soft components to evoke pleasure.
  • Utilize sharp, "dangerous" shapes to increase dynamic tension or provide warning cues, such as hard red shapes.
  • Consistently use deformable interface elements across visual and tactile modalities, particularly suitable for applications like pressure-sensitive buttons, medical, or safety devices.

• Limitations and Future Research Directions:

  1. The study was not applied in real-world interaction scenarios, and the ecological validity of the experiments needs further verification.
  2. Dynamic shape cross-modal correspondence was not included in the study.
  3. Future research could incorporate biometric methods to investigate whether stimuli directly promote emotional arousal.