3DInkGen: Extending Traditional Ink-Painting Artistry with Generative 3D Creation for Novices

Paper Title

3DInkGen: Extending Traditional Ink-Painting Artistry with Generative 3D Creation for Novices

Publication Info

• Topic area: Generative AI for 3D artistic creation in the style of traditional Chinese ink painting.
• Keywords: 3D ink painting, generative AI, style transfer, novice creativity tools, cultural heritage, semantic mapping, 3D Gaussian splatting, user-centered design, digital art, AI-assisted creation.

Background and Problem

• Problem / challenge: Transforming 2D ink paintings into 3D digital forms is complex, requiring expertise in both 3D modeling and traditional ink aesthetics. Existing tools are inaccessible to novices due to high technical barriers and limited stylistic fidelity.
• Significance: Lowering these barriers can democratize 3D ink creation, enabling broader participation in preserving and innovating traditional art forms, with applications in animation, games, and digital media.
• Motivation and related work: Prior efforts in 3D ink painting focus on manual modeling or automated style transfer but fail to balance spatial consistency, stylistic fidelity, and usability for non-experts. AI-driven tools have potential but lack user-centered designs tailored to novices.

Solution

• Proposed approach: 3DInkGen, an AI-powered interactive system, enables novices to create 3D ink artworks by transforming 2D ink elements into editable 3D forms while preserving traditional aesthetics.
• Novelty:
  1. Introduces a four-stage workflow (element extraction, form generation, 3D reconstruction, style transfer) for intuitive 3D ink creation.
  2. Implements a semantic mapping mechanism to align traditional ink painting visual language with generative AI processes.
  3. Demonstrates the use of 3D Gaussian Splatting (3DGS) for lightweight, stylistically consistent 3D representations.
  4. Validates the system through expert evaluations and a user study with novices, showing significant reductions in technical barriers.
• Procedure and key techniques:
  1. Element Extraction: Users extract 2D elements from reference ink paintings using SAM-based segmentation.
  2. Form Generation: Sketch-based interface with ink-style brushes generates 2D forms, enhanced by IP-Adapter for stylistic consistency.
  3. 3D Reconstruction: Converts 2D forms into 3D models using Trellis (3DGS framework) for soft, diffuse ink-like effects.
  4. Style Transfer: Applies stylistic features from master ink paintings to 3D models using a composite loss function tailored for ink aesthetics.

Results

• Concrete findings:
  • 3DInkGen reduced the average creation time to ~2 minutes per artwork, with stages like style transfer taking ~94.6 seconds.
  • Achieved significantly higher scores in creativity (CSI: M=82.06 vs. 63.75), user experience (UEQ: p<0.001 for attractiveness and novelty), and ease of use (USE: p<0.001) compared to the baseline system.
  • Expert evaluations rated 3DInkGen outputs higher in all dimensions (e.g., Brushwork: 4.84 vs. 3.77, p<0.05).
• Advantage over baselines: Outperformed InkBrush (baseline) in usability, creative freedom, and stylistic fidelity. Experts noted superior brushstroke textures, tonal gradations, and structural coherence.
• Experiments / evaluation:
  • Conducted a user study with 16 novices and an expert evaluation with 5 professionals.
  • Participants created 3D ink artworks using both 3DInkGen and InkBrush, with outputs assessed on aesthetics and alignment with traditional ink styles.
  • Quantitative data collected via CSI, UEQ, and USE questionnaires; qualitative insights from interviews and thematic analysis.
• Limitations and future work:
  • Style transfer depends on structural similarity between sketches and references, limiting generalizability.
  • Lacks local editability for combining elements across generations.
  • Limited evaluation of professional users and long-term applications.
  • Future directions include enhancing style-structure alignment, improving local control, expanding user diversity, and adapting the framework to other art forms.

Summary

3DInkGen is an AI-powered system that enables novices to create 3D artworks in the style of traditional Chinese ink painting. By combining semantic mapping, 3D Gaussian Splatting, and style transfer, it simplifies the creation process while preserving artistic fidelity. A user study demonstrated its ability to lower technical barriers, enhance creative freedom, and produce high-quality outputs. Expert evaluations confirmed its effectiveness in retaining traditional ink aesthetics. While limitations in generalizability and local control remain, 3DInkGen offers a promising framework for democratizing 3D artistic creation and preserving cultural heritage.