Creative Blends of Visual Concepts

Research Background and Issues

• What problems or challenges did the authors identify?

  • Visual blending is a powerful design and expression technique that conveys abstract ideas by merging elements from two distinct visual concepts. However, designing effective and visually appealing blends faces multiple challenges, such as selecting appropriate elements, ensuring harmony during the blending process, and transforming abstract concepts into concrete visual representations.
  • Current AI-driven image generation technologies can produce images from text but struggle with abstract descriptions, often resulting in distorted or semantically unreasonable outputs. Additionally, cross-modal relationships related to abstract concepts remain underexplored.

• Why is this issue important?

  • In the design domain, visual blending not only effectively engages audiences but also conveys complex and abstract concepts through visualization. This approach has broad applications in advertising, data visualization, and artistic creation, yet producing high-quality visual blends is often time-consuming and technically demanding.
  • The ability to visualize abstract concepts can greatly expand design possibilities and enhance creativity, especially with the support of AI technologies.

• Research Motivation and Related Work

  • Previous studies have emphasized the importance of visual blending and image metaphors in creative expression, but existing methods primarily focus on combining local shapes or single analytical approaches. They fail to fully explore the potential of multi-element blending and lack systematic tools to support designers in handling visual metaphors during the creative process.
  • This study proposes a method that integrates metaphor theory with AI capabilities to generate blended works that are both abstractly expressive and visually appealing.

Solution

• What methods or solutions did the authors propose?

  • The authors introduced an AI-assisted design system called “Creative Blends,” which connects abstract concepts and concrete objects through metaphors.
  • The system employs a multi-stage pipeline design, including abstract semantic analysis, relevant object identification, and fusion scheme generation based on object and attribute similarity. Visual blending is then achieved using generative text-to-image techniques.

• What are the innovative aspects of this solution?

  • Combines metaphor theory with a commonsense knowledge base to explore associations between objects and their attributes.
  • Provides an analysis method based on similarity and sentiment scores, enabling designers to flexibly select multiple visual blending schemes.
  • Supports rapid generation of diverse design prototypes and allows users to iteratively optimize results.

• What are the implementation steps and key technologies used?

  1. Concept Recognition and Object Enumeration: Using language models (LLMs) and knowledge bases to identify metaphorical relationships between abstract concepts and physical objects, while extracting object attributes.
  2. Similarity and Sentiment Analysis: Utilizing the CLIP model to calculate similarity between objects and attributes, and employing sentiment analysis tools (DistilBERT) to evaluate positive and negative emotions.
  3. Fusion Scheme Generation: Generating blending suggestions through pre-defined prompt engineering and inputting them into text-to-image generation models (e.g., DALL·E 3).
  4. Result Visualization and Exploration: Displaying blending results in an interactive interface, allowing users to explore and compare different design options based on similarity.

Research Outcomes

• What specific outcomes were achieved?

  • Creative Blends significantly improved user efficiency and output quality during the visual blending creative phase. Users could easily iterate and explore various design directions using the system.
  • Experimental results demonstrated that Creative Blends outperformed baseline systems in user experience, creative support, and metaphorical expression.

• What advantages does it have compared to existing solutions?

  • Creative Blends expands the scope of visual blending design, enabling more complex combinations of objects and attributes rather than simple shape overlays.
  • Focuses on the visualization of abstract concepts, addressing distortion issues caused by abstract descriptions in existing AI generation technologies through a metaphor-based approach.

• What were the experimental or evaluation results?

  • User Study: 24 participants compared Creative Blends with conventional AI tools (Google Search combined with ChatGPT). Results showed that Creative Blends significantly outperformed baseline tools in system usability, output quality, diversity, and support for creative exploration.
  • Quantitative Analysis:
    • Creative Blends scored significantly higher on the System Usability Scale (SUS) compared to baseline systems.
    • In the NASA Task Load Index evaluation, Creative Blends effectively reduced users’ mental workload, physical effort, and design effort while increasing satisfaction.
    • In the Creative Support Index (CSI) survey, users reported that Creative Blends better facilitated exploration, collaboration, process enjoyment, and outcome value.

• Limitations and Future Directions

  • Current evaluations are limited to the creative design phase and do not address implementation or real-world application contexts; future research could explore the system’s application across the full design workflow.
  • While the system supports generating multiple blending schemes, users have limited ability to modify generated results; future studies could enhance editing flexibility to further reduce trial-and-error costs.
  • Comprehensive analysis of generated results in terms of style, layout, etc., is lacking; future research could investigate the relationship between user engagement and output quality, optimizing prompt formats and image generation techniques.