Feeling Colours: Investigating Crossmodal Correspondences Between 3D Shapes, Colours and Emotions

Title of the Paper

Feeling Colours: Crossmodal Correspondences Between Tangible 3D Objects, Colours and Emotions

Paper Information

• Domain: Human-Computer Interaction (HCI), Crossmodal Perception, Emotion Research
• Keywords: Multisensory Interaction, Crossmodal Correspondences, Colour, Haptics, Emotion, 3D Printing, Bouba/Kiki Effect

Research Background and Problem

• Identified Problems or Challenges: Current HCI research is increasingly integrating multiple sensory modalities, yet there is a lack of systematic analysis on how crossmodal correspondences (CCs) can be leveraged to design multisensory interaction experiences, especially for non-visual or non-auditory senses like touch. Traditional CC studies often focus on 2D shapes, leaving the relationships between complex 3D shapes, colours, and emotions underexplored.
• Importance of the Problem: Understanding the relationships between sensory modalities and how they jointly influence users' emotional interpretations and experiences is crucial. This understanding can inspire the design of richer and more immersive interactive technologies.
• Motivation and Related Work:
  • The potential of multisensory experiences in cognitive science and HCI remains underutilized.
  • The Bouba/Kiki effect (associating round shapes with soothing sounds and sharp shapes with intense sounds) has been widely applied in 2D visual contexts, but its application in 3D tactile contexts and corresponding design principles remain uncertain.
  • Identifying how complex 3D object characteristics evoke specific visual and emotional responses is essential for expanding the design space and enhancing user experiences.

Proposed Solution

• Proposed Solution: A design framework is proposed to experimentally validate crossmodal relationships between touch (3D shapes), colour, and emotion:
  • Using 3D-printed objects inspired by the Bouba/Kiki effect to explore the tactile properties of objects (complexity and sharpness) and their correspondences with colour (hue and brightness) and emotion (valence, arousal, dominance).
  • Quantifying the mappings between touch, colour, and emotion to propose a design space model.
• Innovations:
  • Extending 2D visual perception research to 3D tactile contexts, integrating visual colour and emotional studies.
  • Providing a crossmodal measurement method based on internal consistency to improve the reliability of correspondence evaluations.
  • Moving beyond traditional visual research limitations to explore more natural tactile interaction scenarios (active tactile exploration).

Research Methodology

Experiment Overview

• Participants: 30 adults with normal vision (all using their right hand for object exploration).
• Experimental Tasks:
  • Colour-Touch Association Task: Evaluating the associations between touch and colour (hue and brightness).
  • Emotion-Touch Association Task: Testing the relationships between 3D shapes and valence, arousal, and dominance.
• Equipment and Materials: A custom object rotation device and 18 3D-printed Bouba/Kiki shapes (varying in sharpness and complexity).
• Data Collection and Analysis Methods:
  • Chi-Square tests for colour evaluations.
  • Two-way ANOVA and trend analysis for emotion rating evaluations.
  • Qualitative data analysis combined with thematic analysis to extract participants' association strategies.

Research Findings

• Specific Findings:
  • Colour-Touch Associations:
    • Round and simple objects strongly associated with blue and high brightness.
    • Sharp and complex objects associated with red and low brightness.
  • Emotion-Touch Associations:
    • Round objects were perceived as more pleasant, controlled, and calming.
    • Sharp objects evoked higher arousal but lower valence and dominance.
  • Qualitative results revealed that participants used geometric features, associations with everyday objects, and valence as primary strategies for forming associations.
• Comparative Advantages:
  • Empirical evidence confirmed significant differences in crossmodal correspondences between 3D tactile properties and 2D visual stimuli. For instance, the association between round shapes and blue manifests through different pathways in touch and vision.
  • The study introduced crossmodal transition characteristics, revealing that object features connect not only within sensory domains but also across physical properties and emotional dimensions.
• Limitations:
  • Small sample size, with participants limited to a single institution.
  • The combined effects of colour and emotion were not systematically explored.
• Future Directions:
  • Expanding the sample size and diversity to improve the generalizability of results.
  • Investigating the combined effects of touch and colour on emotion and validating the applicability of the design space model.
  • Exploring how users systematically or intuitively employ crossmodal correspondences in different interaction contexts.

Design Space and Applications

• A systematic design framework is proposed, linking the complexity and sharpness of tactile shapes with visual colour (hue and brightness) and emotional categories (valence, arousal, dominance) to form design goals and recommendations. For example:
  • Round/low-complexity objects + high brightness/blue = calmness/pleasantness/control.
  • Sharp/high-complexity objects + low brightness/red = arousal/unpleasantness/lack of control.
• Potential Applications:
  • Multisensory storytelling, such as enhancing immersion in virtual reality or emotional learning experiences in education.
  • Assistive technology design, helping users with diverse sensory needs to interact effectively.
  • Creative interactive devices, offering emotional triggers through the fusion of touch and colour.

Conclusion

This study systematically explores, for the first time, the crossmodal correspondences between tangible 3D objects, visual colours, and emotions through the "Bouba/Kiki" effect. It preliminarily constructs a design space based on tactile, visual, and emotional dimensions. The findings provide theoretical support for designing more vivid and engaging multisensory interaction technologies and propose multiple directions for future research and applications.