DesignWeaver: Dimensional Scaffolding for Text-to-Image Product Design

Research Background and Problem

• Problem or Challenge: The authors identified a critical issue in the application of generative AI in product design: beginners often lack domain knowledge and design language expertise. This limits their ability to effectively explore the design space. Additionally, beginners struggle with crafting accurate and meaningful text prompts, leading to generated images that do not align with their expectations.
• Significance: With the advancement of generative AI (e.g., text-to-image generation models), the creative design process has undergone profound changes. These tools can rapidly generate visual content but require skilled prompt engineering to unlock their potential. This poses a barrier to entry for many general designers and beginners who wish to leverage AI.
• Research Motivation and Related Work: Through interviews with 12 experienced designers, the authors found that visual representations (e.g., images and templates) are an essential medium for designers to collaborate with non-expert clients, proving more effective than textual descriptions. This inspired the authors to develop a method to help beginners generate more precise text prompts based on visual feedback. Related work includes explorations of generative AI in design, such as improving prompt engineering, UI optimization, and multimodal interaction support tools. However, there remains a lack of structured support for specific design dimensions.

Solution

• Method or Solution: The authors proposed an AI-driven design interface called “DesignWeaver,” which employs an innovative strategy called “Dimensional Scaffolding” to help beginners understand design language and enhance their design expression capabilities.
• Innovations:
  • Provides a “Dimension Panel” that allows users to extract and add design dimensions such as geometry, style, color, and material based on existing images.
  • A bidirectional interaction mechanism: text prompts generated by users can create images, and the generated images can reveal relevant design dimensions. This structured approach helps users improve their design understanding and iterative capabilities.
  • Highly integrated design process: Users can optimize results by dragging and dropping tags, customizing dimensions, and dynamically modifying prompts.
• Implementation Steps and Techniques:
  1. Users upload initial design documents (e.g., client requirements and reference templates), and the system extracts key design dimensions and their tags using GPT-4.
  2. Users select or modify tags through the “Dimension Panel” to generate longer and more detailed text prompts.
  3. Prompts are used to generate images via OpenAI’s DALL-E 3 API, and users can iteratively optimize prompts.
  4. A model supported by GPT-4o-mini analyzes the generated images and extracts new design dimensions to update the Dimension Panel.
  5. Users complete the design process by scrolling through, comparing, and selecting their preferred design images.

Research Outcomes

• Key Findings:
  • DesignWeaver significantly improved the length and complexity of user prompts, resulting in designs with broader coverage of design language.
  • It enhanced the diversity and creativity of generated images, enabling participants to discover and utilize new design dimensions.
  • Although user prompts became more precise and detailed, experiments revealed that current generative AI models still face technical limitations in handling complex prompts.
• Advantages Over Existing Solutions:
  • Compared to standard text-prompt interfaces, DesignWeaver significantly improved beginners' ability to explore the design space.
  • Images generated by participants were rated as more novel by experts.
  • The tool was particularly effective in helping beginners master design language and perform iterative optimization.
• Experimental or Evaluation Results:
  • In an experimental study involving 52 participants, images generated by users of DesignWeaver received higher novelty scores from experts (average 4.09) compared to the baseline group (average 3.54).
  • The CLIP model was used to evaluate the semantic diversity of generated images; images from the DesignWeaver group exhibited greater semantic variation than those from the baseline group.
  • Participants used more unique design terms to enrich their prompts in DesignWeaver, resulting in the creation of more new dimensions (e.g., “durability” and “visual appeal”).
• Limitations and Future Directions:
  • Existing generative AI tools face technical challenges in interpreting complex text prompts and generating precise images, requiring further optimization.
  • The study focused solely on the furniture design domain (dining chair design); future work could expand to other design fields such as fashion design, architectural design, etc.
  • More research is needed to customize tools for users with varying levels of experience, particularly in balancing the needs of beginners and professional designers.
  • Future work could explore collaborative scenarios, enabling teams to use dimensional scaffolding for co-design processes while incorporating dynamic adjustments and feedback mechanisms.

This structured research approach and interactive design provide critical guidance for the application of generative AI in product creative design and lay a foundation for further research.