Ink Restorer: Virtual Restoration of Ancient Chinese Paintings Inheriting Traditional Restoration Processes

Research Background and Issues

• Identified Problems or Challenges: Due to internal and external factors such as natural aging or poor preservation, many ancient Chinese paintings have suffered severe damage, including cracks, missing parts, and color fading. Traditional painting restoration processes are complex and time-consuming. Moreover, the high professional threshold of this skill makes it difficult for the public to experience or deeply understand the restoration process. Additionally, there are fewer than 100 professional painting restorers in China, posing a risk of the skill being lost over time.
• Importance of the Issue: The restoration of ancient paintings plays a critical role in the preservation and inheritance of Chinese culture. While the public shows significant interest in the restoration process, their understanding of the related knowledge is minimal. Enhancing the public's understanding and experience of the restoration process can not only promote cultural preservation but also catalyze public education and participation, thereby advancing the inheritance of cultural heritage.
• Research Motivation and Related Work: Many existing tools explore AI-assisted painting restoration, but they primarily focus on the authenticity and coherence of the generated results, neglecting the integration of traditional restoration processes and cultural contexts. Given the public's strong interest in the cultural value of painting restoration, it is necessary to bridge the gap between experts and the public in terms of restoration knowledge and experience through technological design.

Solution

• Proposed Method or Solution: The authors designed a virtual restoration tool named "Ink-Restorer," aimed at allowing ordinary users to experience the four stages of the restoration process for ancient Chinese paintings: "Washing" (Xi), "Unveiling" (Jie), "Patching" (Bu), and "Completing" (Quan). The tool integrates AI methods such as image segmentation (SAM), generative techniques (Stable Diffusion), and refinement (LoRA).
• Innovative Aspects of the Solution:
  1. Integrating the four traditional restoration stages into the user experience design to enhance interactivity and cultural relevance.
  2. Combining advanced AI technologies to simplify restoration operations while maintaining process transparency, avoiding the "black box" issue.
  3. Specifically designed for novice users, the tool lowers the learning curve and shortens restoration time through visual guidance and interactive operations.
• Implementation Steps and Key Technologies:
  1. User Interface Design: The tool provides a step-by-step instructional interface, introducing details of the paintings and the restoration process. Users can select paintings of interest and perform restoration, with each stage equipped with specific tools (e.g., AI generation tools and drawing tools).
  2. AI Technology Applications:
     • Using the SAM model for pixel-level selection of damaged areas, offering iterative click functionality to optimize user selection.
     • Applying Stable Diffusion and LoRA techniques for image completion and refinement, further adapting to the style of Chinese paintings.
  3. User Operations and Generation Monitoring: By customizing the restoration workflow and AI models, the tool tracks user operations and performs automated result optimization.

Research Outcomes

• Specific Results:
  • Ink-Restorer significantly enhanced users' cultural understanding and recall of the restoration process.
  • The tool's restoration quality (including color, detail handling, and artistic similarity) was notably superior to two baseline tools.
  • Experimental results demonstrated outstanding performance in user learning, efficiency, and satisfaction, particularly bridging the skill gap between experts and ordinary users.
• Advantages Compared to Existing Solutions:
  1. Cultural Inheritance: Unlike purely automated AI tools, Ink-Restorer emphasizes integration with traditional restoration steps, allowing users to both experience the process and perceive the cultural significance behind it.
  2. Interactive Adaptability: The tool enables users to enhance refinement results through AI assistance while supporting manual creation, increasing user engagement and control.
  3. Educational Applicability: As an easy-to-learn and user-friendly educational tool, it is suitable for use in museums and classroom teaching.
• Experimental or Evaluation Results:
  • Quantitative analysis of the User Experience Questionnaire (UEQ) and Usability Scale (USE) showed that Ink-Restorer significantly outperformed baseline tools in terms of attractiveness, efficiency, and satisfaction.
  • Expert evaluations indicated that the restoration effects achieved using Ink-Restorer were closer to the original paintings, with superior color and detail generation.
• Limitations and Future Directions:
  1. Material Realism: The tool does not currently address the issue of matching specific paper and texture required in actual restoration. Future research could focus on developing algorithms to generate restoration materials consistent with the texture of the paintings.
  2. Detail Optimization: There is a gap in precision between AI-generated details and the actual damaged textures. Further optimization of segmentation algorithms (e.g., SAM) could improve the tool's ability to handle fine-grained damage.
  3. Cross-Domain Applications: The tool could be extended to other types of artwork (e.g., oil paintings, Indian Tanjore paintings) and virtual reality environments in the future to enhance immersive cultural interaction experiences.

The research on Ink-Restorer provides valuable insights into bridging the cultural value of traditional art restoration with modern digital technology, driving digital innovation and transformation in the preservation and inheritance of artistic heritage.