InkIdeator: Supporting Chinese-Style Visual Design Ideation via AI-Infused Exploration of Chinese Paintings

Paper Title

InkIdeator: Supporting Chinese-Style Visual Design Ideation via AI-Infused Exploration of Chinese Paintings

Publication Info

• Topic area: AI-assisted tools for cultural and creative visual design
• Keywords: Chinese painting, visual design, generative AI, cultural symbols, ideation support, design space, multimodal models, creativity tools, user study, cultural heritage

Background and Problem

• Problem / challenge: Designers face difficulties in efficiently searching, analyzing, and integrating cultural symbols, emotions, compositions, and styles from Chinese paintings into their work due to scattered resources, lack of domain knowledge, and inefficiencies in visualizing ideas.
• Significance: Addressing these challenges can enhance the creative process for Chinese-style visual design, promote cultural transmission, and lower barriers for non-experts to engage with traditional Chinese aesthetics.
• Motivation and related work: Prior tools like MetaMap and HarmonyCut support example-based ideation but do not address the unique challenges of Chinese-style design, such as the cultural and aesthetic complexity of Chinese paintings. This paper builds on these gaps by focusing on ideation with Chinese paintings.

Solution

• Proposed approach: InkIdeator, an AI-powered ideation support tool that facilitates Chinese-style visual design by leveraging a dataset of annotated Chinese paintings and generative AI capabilities.
• Novelty:
  1. Development of a structured dataset of 16,315 Chinese paintings annotated with cultural symbols, emotions, compositions, and styles.
  2. Introduction of a multi-panel interface (Symbol Association, Image Library, Moodboard, Image Generation) to support ideation.
  3. Integration of generative AI for visualizing ideas and exploring design dimensions.
  4. Evaluation through user studies and extended use cases with experienced Chinese painters.
• Procedure and key techniques:
  • Dataset creation: Crawling and annotating 16,315 Chinese paintings using multimodal large models and feedback from Chinese painters.
  • Interface design: Four panels for symbol suggestion, example search, dimensional analysis, and image generation.
  • Generative AI integration: GPT-4o and MidJourney for generating design intentions, images, and poems.
  • User evaluation: Conducting a within-subjects study and extended use cases to assess effectiveness.

Results

• Concrete findings:
  • Annotated dataset includes 1265 cultural symbols, 4903 emotional concepts, and rich keywords for styles, compositions, brushstrokes, and color tones.
  • InkIdeator significantly outperformed the baseline in supporting organized exploration (p=0.006), extracting design elements (p=0.031), and visualizing ideas (p=0.021).
  • Generated images were rated as more relevant to design intentions compared to baseline (p=0.035).
• Advantage over baselines:
  • Enhanced exploration of design space with annotated keywords.
  • Improved user experience in ideation, with higher ratings for expressiveness, immersion, and ease of idea exploration.
  • Generated images better aligned with user-selected keywords and cultural aesthetics.
• Experiments / evaluation:
  • Within-subjects study with 12 participants comparing InkIdeator to a baseline system.
  • Extended use cases with two experienced Chinese painters to explore generalizability.
  • Evaluation metrics included clarity and appeal of ideas, user experience (CSI, NASA-TLX), and usefulness of system panels.
• Limitations and future work:
  • Small sample size and time constraints in user studies.
  • Challenges in generating culturally accurate images due to biases in generative AI models.
  • Lack of manual sketching support and potential for user over-reliance on AI outputs.

Summary

InkIdeator is an AI-powered tool designed to support Chinese-style visual design by facilitating the exploration of cultural symbols, emotions, compositions, and styles in Chinese paintings. It leverages a structured dataset of 16,315 annotated paintings and integrates generative AI for idea visualization. User studies demonstrated its effectiveness in enhancing creative ideation, particularly in organized exploration and efficient visualization of ideas. While promising for cultural design tasks, future work should address limitations in dataset diversity, manual sketching integration, and cultural accuracy of AI-generated outputs.