Inkspire: Supporting Design Exploration with Generative AI through Analogical Sketching

Research Background and Problem Statement

• Research Problems and Challenges:

  1. Current Text-to-Image (T2I) models can generate high-quality images for designers but struggle to interpret abstract language. For instance, generating inspiring designs based on abstract concepts like "protective car" remains challenging.
  2. Text-driven user interfaces often lead to "design fixation," where designers repeatedly generate slight variations of a particular idea or image instead of exploring novel designs.
  3. Outputs generated by T2I models are often "too complete," which can limit designers' creativity and reduce their control over design details during further iterations.

• Importance and Research Motivation:

  1. The creative process of designers requires extensive and dynamic exploration, as well as the ability to address abstract and complex design goals.
  2. Observations of how designers use generative AI suggest that encouraging designers to explore more design options and collaborate with AI for cross-domain inspiration (e.g., analogy-driven design) is highly valuable.
  3. This research aims to integrate T2I models into designers' workflows effectively, providing support for creative design.

• Related Work:

  1. Literature shows that T2I models can enhance the potential for generating new ideas but often limit users' design diversity and originality.
  2. The "high-fidelity image output" in existing fixed workflows tends to cause psychological fixation among designers. Reducing the complexity of abstract images can mitigate design fixation.

Solution

• Proposed Solution:

  1. Develop an innovative design tool called Inkspire, driven by sketches to support iterative design and exploration.
  2. Integrate an "analogy inspiration" feature to transform abstract design themes into concrete, visual inspiration materials (e.g., associating "protective" with turtles, armor, or fortresses).
  3. Provide a "design-to-sketch reverse feedback loop," converting high-fidelity designs into low-resolution sketch outlines to help designers avoid fixation and iteratively improve their designs.

• Innovations:

  1. Analogy-Driven Design: Utilizing large language models (LLMs) like GPT-4 to generate multi-domain analogy inspirations related to target concepts (e.g., nature, architecture, fashion).
  2. Sketch-Guided Generation: Employing dynamically adjustable generation controls to create new designs in real-time based on users' progressively refined sketches, while preserving flexibility for exploration.
  3. Sketch Scaffolding: Abstracting AI-generated high-fidelity images into low-resolution sketches to guide subsequent design iterations, thereby reducing the visual fixation effect of "finished products."

• Implementation Steps and Key Technologies:

  1. Sketch2Design Module:
     • Use LLMs (e.g., GPT-4) to generate analogy concepts from abstract to concrete.
     • Implement dynamic guidance scales in ControlNet models to enable sketch-based interactive generation, supporting real-time design updates with each new user stroke.
  2. Design2Sketch Module:
     • Extract key contours from generated designs using semantic segmentation.
     • Apply soft-edge detection techniques to reduce visual details in sketches, creating high-quality yet abstract sketch scaffolding.
  3. User Interface Interaction:
     • Provide an "Analogy Panel" for quick generation of thematic inspirations, a "Sketch Panel" for iterative sketch-based design, and an "Evolution Panel" to record design history.

Research Outcomes

• Specific Results:

  1. Inkspire outperforms traditional T2I tools (e.g., ControlNet) in the exploration and inspiration phases. Designers found it more effective in stimulating creativity and enhancing design diversity.
  2. Introduced a new sketch-driven, analogy-centered design workflow, enabling smoother AI collaboration.

• Advantages Over Existing Solutions:

  1. Supports genuine "human-AI co-creation," improving scores for designer-AI "communication" and "partnership."
  2. Designers feel a stronger sense of control and ownership over the final designs.
  3. Effectively reduces design fixation behavior, allowing more novel design directions to be explored.

• Experimental or Evaluation Results:

  1. Exploration and Inspiration: Users' Creative Support Index (CSI) in Inkspire was significantly higher than the baseline control group, especially in exploration (p<0.01) and inspiration (p<0.01).
  2. Sketch Behavior: Users of Inkspire made fewer sketch strokes (μ=17.3) that were more abstract and "lightweight," facilitating smoother iterative processes compared to baseline tools.
  3. Human-AI Collaboration: User self-assessments showed significant improvements in AI control, communication, and partnership when using Inkspire.

• Limitations and Future Directions:

  1. Inkspire is primarily focused on the early stages of design exploration, with limited support for later-stage refinement.
  2. While the analogy generation feature performs well in inspiring design diversity, it does not yet support more complex multi-branch explorations.
  3. System functionalities (e.g., sketch tools, semantic precision controls) require further improvement. Future enhancements could include adding more complex design constraints (e.g., physical or material characteristics).

  Potential Directions include exploring cross-domain applications (e.g., illustration or architectural design), introducing stronger multi-branch exploration support for team collaboration, optimizing user experience, and expanding the experimental sample size to validate long-term effects.