GenColor: Generative Color-Concept Association in Visual Design

Research Background and Issues

• Identified Problems or Challenges:
  Traditional methods of associating colors with concepts rely on query-based image references and color extraction from images. However, these methods perform poorly when dealing with uncommon concepts or contextual associations. For example, colors related to specific styles or semantic contexts are often inaccurately captured. Additionally, the retrieved images are susceptible to variations in image quality, lighting, and style, leading to inaccurate or unstable color associations.

• Significance:
  Color is a powerful expressive tool in visual design. By correctly matching colors with semantics, the effectiveness of design communication can be enhanced. In fields such as information visualization and graphic design, color associations improve the efficiency of information classification and emotional communication.

• Research Motivation and Related Work:
  Current computational methods that utilize large-scale text corpora or image databases can automate color association generation but suffer from insufficient semantic context processing and unstable image reference results. Moreover, designers have higher demands for context-dependent color combinations and the coordination of primary and secondary colors.

Solution

• Proposed Method or Solution:
  The study introduces GenColor, a framework that integrates generative AI to extract semantically relevant colors from images generated by text-to-image (T2I) models. The framework consists of three stages:

  1. Conceptual Instancing: Using Stable Diffusion to generate image samples closely related to specified concepts and contexts.
  2. Text-guided Image Segmentation: Employing semantic segmentation techniques to precisely extract image regions associated with the concept.
  3. Color Association: Extracting a color palette composed of primary and secondary colors from the segmented regions.

• Innovations:

  1. Generating context-dependent images through generative models addresses the stability and flexibility limitations of existing query-based methods.
  2. High-quality images are produced using diffusion models, and segmentation techniques focus on concept-related regions within the images, improving the accuracy of color extraction.
  3. A novel color extraction method combining color grouping and clustering is proposed, providing an intuitive palette representation with clear primary and secondary color distributions.

• Implementation Steps and Key Techniques:

  1. Prompt Design and Generation Strategy: Crafting refined prompts (e.g., descriptions of associated contexts) to control the quality and diversity of generated images.
  2. Segmentation Processing Based on GroundingDINO and SAM: Precisely segmenting concept-related parts from generated images, avoiding semantic deviation caused by background removal.
  3. Palette Generation: Creating palettes using color clustering techniques, with primary colors placed at the center and secondary colors surrounding them to enhance designers' intuitive perception.

Research Outcomes

• Specific Results:
  The GenColor framework can generate color combinations that align more closely with designers' selections, particularly excelling in handling environmental concepts and context-related ideas. The study also provides a foundational dataset of designer hand-drawn baselines for quantitative evaluation.

• Comparison with Existing Solutions and Advantages:

  1. In terms of alignment with designer cognition, GenColor's results are more representative than query-based methods (e.g., photo or clip art-based approaches).
  2. In color difference evaluation, CIEDE2000 scores show that the generative method significantly outperforms traditional query methods in context-dependent concepts.
  3. User studies reveal that GenColor's results achieve higher ratings in "representativeness" and "preference," especially in complex scenarios.

• Experimental or Evaluation Results:

  • In representativeness experiments, the scores for generated images and clip art were 4.55 and 4.71, respectively, higher than those for query-based images (4.61) and clip art (4.68).
  • For contextual concepts, the generative method exhibited lower color difference (CIEDE2000 for generated clip art was 13.37, compared to 27.08 for query-based clip art).
  • User studies showed that users consistently rated the generated colors higher in representativeness and preference for contextual concepts compared to traditional methods.

• Limitations and Future Directions:

  1. Challenges in Handling Abstract Concepts: The current method focuses on concrete concepts, and future work should explore color representations for abstract concepts such as "hope" or "fear."
  2. Context Expansion: This study primarily focuses on conditional and emotional contexts. Future research could explore broader contexts, such as audience preferences and cultural backgrounds.
  3. Bias in Generative Models: Existing T2I models may exhibit biases in global contexts (e.g., color representation of currencies), necessitating further research to mitigate these biases and enhance global applicability.

Conclusion

GenColor innovatively integrates generative AI technology to propose and validate a flexible and robust framework for associating colors with concepts, effectively supporting designers' workflows. Future directions include exploring capabilities for handling multiple concepts and more complex scenarios, as well as enhancing personalized and culturally relevant color suggestions.