Exploring Interactive Color Palettes for Abstraction-Driven Exploratory Image Colorization

Document Title

Exploring Interactive Color Palettes for Abstraction-Driven Exploratory Image Colorization

Document Information

• Subject Area: Human-Computer Interaction Design and Visual Computing
• Keywords: Color Design, Image Colorization, Creative Design, Visual Abstraction, AI Models

Research Background and Problem

• Problem or Challenge: Current image colorization tools (e.g., Photoshop) primarily adopt solutions based on specific image region operations, focusing on precise final product creation. However, they lack support for the exploratory phase in early design, limiting users' creative divergent thinking.

• Significance: Color design plays a critical role in fields such as graphic design, product design, interior design, and fashion, influencing the overall atmosphere, emotional expression, and audience reception of visual content. Enhancing exploratory workflows in the early design phase can significantly boost creativity.

• Research Motivation and Related Work:

  1. Abstraction-driven approaches have previously demonstrated potential in fostering creative thinking in shape design and architectural planning, but their application in image color design remains underexplored.
  2. Current research focuses on optimizing the computational accuracy and efficiency of color distribution, neglecting users' understanding of the size and positional relationships of colors in abstract visual information.
  3. The study poses four core questions:
     • RQ1: What are the potential visual abstraction design options suitable for color design?
     • RQ2: What are the advantages and challenges of various abstraction options in user interaction?
     • RQ3: How does abstraction-driven operation influence the ideation process of color design?
     • RQ4: How do different abstraction options affect users' perception of color composition?

Solution

• Proposed Method:

  1. Develop a system named Mondrian that empowers users with interactive color abstraction capabilities.
  2. The system employs three formats of color palettes (1D uniform, 1D proportional, 2D spatial) and integrates AI models to accomplish image recolorization tasks.

• Innovations:

  1. Combines abstraction-driven approaches with artificial intelligence, allowing users to focus primarily on visual abstraction design while AI handles specific image region colorization.
  2. Designs multi-level palette interaction methods (1D and 2D) to support global and local geometric-level color exploration.

• Implementation Steps and Key Techniques:

  1. Palette Extraction:
     • Use k-Means clustering to extract primary colors from images.
     • Generate 2D spatial palettes using the SLIC algorithm, preserving the proportions and pixel positions of colors.
  2. AI Recolorization Module:
     • Optimize color distribution and proportions using the pre-trained HistoGAN model.
     • Employ Zhang et al.'s deep learning method to process spatial associations for high-quality image recolorization.
  3. User Interaction Interface Design:
     • Includes result preview, palette editing, and bookmark tracking panels to support iterative design by users.

Research Outcomes

• Specific Results:

  • The Mondrian system facilitates more creative exploration by users, breaking linear workflows and providing open-ended support for design ideation.
  • Experiments reveal that different abstract palette formats exhibit unique characteristics in terms of intuitiveness, expressiveness, and creative support, such as:
    • The 1D+ format supports proportional refinement.
    • The 2D format is more suitable for images with distinct regions, emphasizing spatial relationships.

• Advantages and Comparisons:

  • Compared to Photoshop: Mondrian is simpler and better suited for early design exploration, supporting non-linear workflows. However, Photoshop excels in precision and complex palette manipulation.
  • Creativity Score Index (CSI) analysis indicates that Mondrian has significant advantages in exploration, immersion, outcome reward, and collaboration convenience.

• Experimental or Evaluation Results:

  1. User Study: Tasks completed by 12 participants show that Mondrian provides greater creative support while highlighting the usage characteristics of each abstract palette.
  2. Survey Study: Systematic evaluation of the aesthetic quality and emotional metrics of color combinations reveals that the integration of proportion and spatial placement significantly impacts human color perception.

• Limitations and Future Directions:

  • Limitations:
    • The current system has a limited range of colors and lacks real-time preview capabilities.
    • The study primarily involves general users unfamiliar with color theory, excluding expert-level designers.
  • Future Directions:
    • Explore more advanced or dynamic palette visualization designs.
    • Introduce hierarchical structures to palettes for finer-grained control.
    • Optimize the mapping mechanism between abstraction and specificity to enhance user design precision.