Style2Fab: Functionality-Aware Segmentation for Fabricating Personalized 3D Models with Generative AI

Title of the Paper

Style2Fab: Functionality-Aware Segmentation for Fabricating Personalized 3D Models with Generative AI

Paper Information

• Research Area: Human-Computer Interaction; Generative AI; 3D Printing Design and Personalization
• Keywords: Personalized Manufacturing, Digital Fabrication, 3D Printing, Generative AI, Functionality-Aware Segmentation, Stylized 3D Models, Thingiverse, Deep Learning, Design Support Tools, Segmentation and Classification

Research Background and Problem Statement

• Identified Problems or Challenges: Generative AI can assist users in modifying 3D models, but current editing methods are often applied globally, which may compromise the functionality of the models. For example, altering functional parts such as the base of a vase could render the model unusable.
• Significance: In the field of 3D printing, the functionality of a design is directly tied to its geometric structure. Editing functional parts carelessly may prevent the object from fulfilling its intended purpose, posing a significant barrier for ordinary users seeking to personalize existing designs.
• Motivation and Related Work: Current CAD tools have the capability to mark functional regions, but users often struggle to identify functional and aesthetic parts in legacy models, especially those lacking metadata. Additionally, existing tools typically require a high level of technical expertise, making them inaccessible to average users.

Proposed Solution

• Method or Solution Proposed: The authors developed a system called Style2Fab, which automatically decomposes 3D models into functional and aesthetic parts, allowing users to modify only the aesthetic parts while preserving the model's functional integrity.
• Innovative Contribution: Based on a functionality classification system derived from the analysis of 1,000 Thingiverse models, the authors proposed a functionality-aware segmentation method combined with generative AI techniques to enable localized stylization of models without affecting their functionality.
• Implementation Steps and Key Techniques:
  1. Functionality Classification: Developed a two-axis classification system for functionality and aesthetics, using qualitative coding and iterative methods to classify models.
  2. Spectral Segmentation Technique: Applied spectral segmentation methods to automatically divide models into segments based on the structure of 3D meshes.
  3. Similarity Analysis: Used topological analysis to classify functional segments, ensuring that functional parts remain unchanged during stylization.
  4. Generative Stylization Technique: Employed CLIP-based Text2Mesh technology to apply stylization transformations exclusively to aesthetic parts.
  5. User Interface Design: Developed a plugin for the open-source tool Blender, enabling users to verify functional segments and selectively apply stylization.

Research Outcomes

• Specific Outcomes:
  1. Functionality Classification Method: Extracted 938 models from Thingiverse and established functionality and aesthetic classification dimensions (external functionality and internal functionality).
  2. Validation of Segmentation Techniques: Demonstrated effective segmentation and classification across models with varying levels of complexity.
  3. GUI Implementation and User Study: Developed a user-friendly interface that allows users to quickly verify and stylize models, accompanied by a user study involving 8 participants. Results showed significant improvements in user efficiency and model classification accuracy.
• Advantages: Compared to existing tools, Style2Fab reduces the need for users to understand model functionality, preserves functional integrity, and minimizes user workload through automated segmentation and classification.
• Experimental/Evaluation Results:
  • The time required for users to classify functional segments was significantly reduced, with improved accuracy.
  • Segmentation results performed particularly well in multi-component models with complex functionality.
  • Users generally reported that the tool enhanced the stylization design process while ensuring functional completeness.
• Limitations and Future Directions:
  1. Limitations in Functionality Definition: Current functionality classification relies solely on topological similarity and does not account for the complexity of functional behavior, such as environmental influences.
  2. Need for Dataset Expansion: The evaluation was based on a relatively small dataset, necessitating larger annotated datasets for optimized classification in the future.
  3. Integration of Deep Learning: Combining topological classification with deep learning methods to handle complex functional structures.
  4. Support for Diverse Applications: Expanding the method to applications in healthcare, education, and complex machinery.

Conclusion

Style2Fab provides 3D printing users with personalized advantages through an innovative functionality-aware segmentation method, reducing the difficulty of design and modification. Although the method has certain limitations, future advancements in dataset expansion and functionality definitions could further drive the application of generative AI in the manufacturing domain.