FontCraft: Multimodal Font Design Using Interactive Bayesian Optimization

Research Background and Problem

• Problem or Challenge: Designing new fonts requires significant labor and expertise, especially when creating large character sets (e.g., GB18030 Chinese fonts), which can take multiple experts over a year. Additionally, current automated font generation methods require pre-designed character samples, posing a challenge for non-professional users.
• Importance: Convenient font generation tools can not only improve design efficiency but also lower the barriers to font design, meeting diverse design needs and better supporting practical applications such as advertisements and posters.
• Research Motivation and Related Work: Many generation model-based methods have been applied to font generation (e.g., DG-Font), but they typically require users to provide pre-designed character samples or have a deep understanding of the generation model, limiting accessibility for non-professional users. Furthermore, existing human-computer collaborative Bayesian optimization methods, while effective, may reduce user creativity and control.

Solution

• Method or Solution: A system called FontCraft is proposed, combining multimodal references (text, images, font files) with human-computer collaborative Bayesian optimization (PBO), enabling users to generate high-quality fonts without pre-designed character samples.
• Innovations:
  1. Multimodal-Guided Subspace: Maps user-provided multimodal inputs into the style latent space of the font generation model, allowing users to express design preferences through text and images.
  2. Traceable Preference Modeling: Provides a history interface that allows users to revert to previous design states, breaking the irreversible workflows of traditional methods.
  3. Style Propagation and Optimization: Users can design the style of a single character and propagate it to the entire character set, with support for further manual optimization.
• Implementation Steps and Key Techniques:
  1. Style Exploration: Users explore the latent font style space using sliders; the system leverages the multimodal capabilities of the FontCLIP model and Bayesian optimization to recommend new candidate styles.
  2. Style Propagation: After designing one character, its style can be applied to other characters, ensuring overall consistency.
  3. History Tracing: A history interface facilitates users in revisiting and refining design choices.
  4. Output Generation: The system generates OpenType Font (OTF) files for users to download.

Research Outcomes

• Specific Achievements:
  • Introduced the first interactive font generation system that does not require pre-designed character samples.
  • Successfully integrated multimodal references with human-computer collaborative Bayesian optimization to enhance user control.
  • Provided style propagation functionality, simplifying the rapid creation of stylistically consistent fonts.
• Advantages Compared to Baselines:
  • Compared to baseline systems (which only provide slider-based exploration), FontCraft generates font styles that better align with user preferences and demonstrates superior style consistency.
  • The history interface and multimodal references significantly enhance user design flexibility and operational experience.
• Experimental or Evaluation Results:
  • Quantitative Evaluation: The generated fonts outperform baseline systems in terms of target similarity and inter-character consistency.
  • User Study: Non-professional users generally affirmed the system's ease of use, particularly highlighting the value of multimodal input and style propagation functionalities.
  • Demonstration: Successfully applied to real-world design scenarios (e.g., conference logos and promotional posters) and showcased support for CJK (Chinese, Japanese, Korean) character design.
• Limitations and Future Directions:
  • Generation Quality: Current generated fonts exhibit some distortion issues (e.g., line warping), limiting their direct application in production.
  • Feature Improvements: Users expressed a desire for direct editing of generated characters and more flexible control over the style propagation process.
  • Latent Space Exploration Improvements: Broader latent space construction through more extensive multimodal references has optimization potential.
  • UI Optimization: Consider adding support for screen typography, character spacing, and other layout features to improve font production quality.

Through FontCraft, this work demonstrates an innovative font design tool aimed at non-professional users, not only lowering the barriers to font design but also addressing multiple shortcomings of existing solutions through technological innovation.