CreativeConnect: Supporting Reference Recombination for Graphic Design Ideation with Generative AI

Document Title

CreativeConnect: Supporting Reference Recombination for Graphic Design Ideation with Generative AI

Document Information

• Subject Area: Graphic Design, Generative Artificial Intelligence, Creative Support Tools
• Keywords: Creative Support Tools, Graphic Design Ideation, Reference Recombination, Machine Learning, Generative AI

Research Background and Problem

• Identified Problems or Challenges:

  1. Graphic designers face significant effort during the creative process of reference recombination, particularly in decomposing elements from references and finding suitable combination strategies.
  2. Novice designers struggle more than professionals to extract inspiration from references and integrate references from different domains.
  3. Existing methods for generating design combinations have limitations, focusing on precise and harmonious combinations while lacking diversity to support exploratory creativity.

• Importance: Creative generation in graphic design often relies on combining existing examples (combinatorial creativity). However, current tools are inefficient in reference decomposition and combination generation, and designers often face constraints in finding creative directions and generating diverse solutions. Addressing these issues could accelerate the design process and enhance design quality and diversity.

• Research Motivation and Related Work: To understand the challenges designers face in reference recombination, this study explores a two-stage creative thinking model (concept ideation stage and visual development stage). The authors propose tool design goals, including element recommendation, diverse combination generation, and output control, to improve efficiency and creativity in reference recombination.

Solution

• Proposed Solution: A generative AI-supported graphic design tool—CreativeConnect—was developed. The system simplifies the reference recombination process through keyword extraction, keyword recommendation, and keyword merging, while providing inspiration via sketch generation and textual descriptions.

• Innovations:

  1. Offers multiple keyword classifications and extraction based on references, covering "thematic content," "actions and poses," "themes and emotions," "layout," etc.
  2. Automates keyword recommendation and recombination to support designers in expanding their thinking.
  3. Utilizes low-fidelity sketch outputs, enabling designers to engage more actively in creative generation.

• Implementation Steps and Key Technologies:

  1. Keyword Extraction: Initial text descriptions are generated using the image captioning model (BLIP-2), followed by keyword extraction via GPT-4.
  2. Keyword Recommendation: Related keywords are generated using GPT-4, expanding references through semantic combinations.
  3. Keyword Recombination and Sketch Generation: Text descriptions are generated using GPT-3.5, combined with layout models (Segment Anything and Layout Diffusion) to produce diverse sketches, which are converted into simple line sketch formats.

Research Outcomes

• Results:

  1. CreativeConnect significantly enhances designers' ability to identify reference elements and generate diverse creative ideas.
  2. Compared to baseline systems, users of CreativeConnect can quickly generate more creative sketches and perceive their designs as more unique and efficient.
  3. Experimental results demonstrate that the system supports users in exploring combination possibilities more comprehensively during reference recombination, unlocking creative potential.

• Advantages:

  1. Integrates generative AI technology to produce diverse combinations of reference resources.
  2. Uses sketches instead of finished images, avoiding overly detailed outputs that could limit user creativity.
  3. Provides a user-friendly interface, making dynamic interactions between keywords and layouts a core process for creative inspiration.

• Experiment or Evaluation Results: User studies (N=16) show that CreativeConnect outperforms baseline systems in identifying reference elements and generating diverse creative ideas. Participants were more willing to experiment with system-suggested combination options and add personal creativity based on sketch results.

• Limitations and Future Directions:

  1. The current study focuses only on novice designers; future research could expand to professional design scenarios or other design domains.
  2. Functions supporting different creative stages (from exploration to implementation) require further optimization.
  3. The short duration of experiments limits observations of long-term design behaviors; the tool's long-term effectiveness should be validated in real-world projects.