Personalizing Products with Stylized Head Portraits for Self-Expression

Title of the Paper

Personalizing Products with Stylized Head Portraits for Self-Expression

Paper Information

• Subject Area: Human-Computer Interaction (HCI); Personalized Design and Stylized Image Generation.
• Keywords: Stylized portraits, artificial intelligence, design support tools, self-expression, human-computer interaction, product personalization, deep learning, design and creativity tools.

Research Background and Problems

• Problems and Challenges:

  • Product personalization as a form of self-expression has garnered significant attention, but most studies focus on embedding abstract data (e.g., numbers or behavioral records) rather than non-abstract personal data (e.g., facial photos).
  • The process of generating portraits by combining facial photos with artistic styles involves visual style transfer, which presents complex challenges in terms of matching visual characteristics, controlling generation quality, and addressing ethical concerns.
  • Current generation models, such as GANs or CLIP-based models, perform poorly when style transfer involves large domain gaps (e.g., from real photos to illustrations), and high-quality paired datasets suitable for training are extremely scarce.

• Significance:

  • Portraits are an important visual symbol of personal identity. If widely applied to objects in a stylized manner, they could imbue these objects with stronger personal meaning and uniqueness, fulfilling users' aesthetic needs and desires for self-expression.

• Research Motivation and Related Work:

  • Propose new design-oriented support tools to address technical and design challenges in transforming facial photos into stylized illustrations, creating vectorized portraits ready for direct printing.
  • Apply stylized portraits to the personalized design of physical products, advancing practical applications in terms of process efficiency, output quality, and user experience.

Solution

• Proposed Solution:

  • Designed a support tool named PicMe, which generates vector-format stylized portraits from user facial photos and allows users to adjust decorations and preview how the image would appear on a specific product.
  • This solution not only supports portrait generation but also provides interactive tools for adding decorative elements (scenes) and previewing the overall packaging effect.

• Innovations:

  • Proposed a phased deep learning algorithm to narrow the domain gap between real photos and illustration styles, thereby improving the quality of generation in the stylized portrait design process.
  • Addressed the scarcity of training data by expanding the existing Open Peeps illustration dataset.
  • Provided tools to assist users in designing personalized products, enhancing self-expression.

• Implementation Steps and Core Techniques:

  • The image stylization generation algorithm is implemented through four modules:
    1. Estimator: Converts user photos into preliminary, roughly stylized bitmap images.
    2. Composer: Selects suitable image components from the illustration dataset to generate stylized portrait illustrations.
    3. Encoder: Uses the CLIP model to map photos and illustrations into the same latent embedding space.
    4. Search Engine: Finds and recommends illustration combinations in the latent space that closely match the user's photo.
  • The user interface provides three main functional pages: portrait generation page, scene selection page, and product preview page.
  • Dataset expansion: Professional designers created decorative scenes and detailed facial components (e.g., hairstyles and expressions) to enhance the diversity of generated results.

Research Outcomes

• Specific Outcomes:

  • The PicMe system can generate stylized portraits for users and provide satisfactory design tools in subsequent product personalization stages, enhancing users' self-expression experience.
  • Experiments demonstrated the algorithm's generalizability to non-abstract data (e.g., facial images) and various illustration datasets.

• Advantages Over Existing Solutions:

  • The phased deep learning algorithm effectively addresses the domain gap issue, producing high-quality results.
  • Supports the generation of vector-format graphics, suitable for real-world printing scenarios and adaptable to different materials and product sizes.
  • Users can enhance their design expressiveness through scene decoration and preview adjustments.

• Experimental or Evaluation Results:

  • Experiment 1 (Effectiveness Validation): Compared multiple baseline models and alternative algorithm modules using 20 validation images, with MAP metrics used to measure generation quality.
    • PicMe achieved the highest generation accuracy, demonstrating the effectiveness of the module design.
  • Experiment 2 (Generalization Validation): Experiments with different open-source illustration datasets (Avataaars and Avatar Illustration System) showed strong adaptability of the algorithm, enabling diverse stylized results.
  • User Study:
    • 40 participants unanimously found the tool intuitive and easy to use, providing positive feedback on the quality of generated portraits and the tool's creative support.
    • Personalized products helped users express individuality, increasing attention and social connections.

• Limitations and Future Directions:

  • The current dataset primarily focuses on cartoon-style illustrations; future work could expand to more artistic styles or dynamic representations.
  • Model training needs to enhance transfer learning capabilities to reduce the retraining cost for different style transfer tasks.
  • Explore stronger intelligent user preference prediction capabilities to assist in refining the human-computer collaboration experience.