CalliSence: An Interactive Educational Tool for Process-based learning in Chinese Calligraphy

Research Background and Issues

• Identified Issues:

  1. Beginners learning Chinese calligraphy often focus solely on the final appearance of the work, neglecting critical brushstroke details and writing rhythm.
  2. Teachers find it challenging to clearly explain complex brushstroke techniques, especially dynamic aspects such as finger pressure changes, brush angle, and tip movement, which further hinders students' understanding of the writing process.
  3. During classroom teaching and review, students often forget brushstroke details after completing their writing, making it difficult to align with the feedback provided by teachers.

• Significance:

  1. Chinese calligraphy is a vital part of intangible cultural heritage, and its artistry and techniques are preserved through the transmission of brushstroke methods.
  2. Writing techniques involve not only static results but also complex dynamics of force, skill, and rhythm, which are essential to the essence of the art form.
  3. In modern education, the transmission of calligraphy faces challenges due to a lack of effective technological tools to support learning.

• Research Motivation and Related Work:

  1. Motivation: Address issues such as teachers' inability to clearly convey brushstroke, finger pressure, and rhythm details, as well as the inefficiency of beginners in mastering fundamental skills.
  2. Related Work:
     • Existing technologies primarily focus on recording static calligraphy forms (e.g., optical tracking, painting robots) while neglecting dynamic processes.
     • Visual feedback and kinesthetic learning have been proven effective in other fields (e.g., dance, music performance, or golf), but their application in calligraphy education remains limited.
     • Current haptic feedback devices or mechanical pen simulation technologies are expensive and fail to replicate the texture and experience of traditional brushes.

Solution

• Methods and Tools:

  1. Proposing an interactive teaching tool called "CalliSense," which integrates low-cost devices (e.g., smartphones, pressure sensors, and inertial sensors) to capture the calligraphy writing process.
  2. The system captures writing dynamics (e.g., brush rotation and tilt, finger pressure, writing rhythm) and aligns them precisely with the resulting calligraphy forms, presenting them intuitively through a visual interface.

• Innovations:

  1. The system achieves fine-grained dynamic data collection of brushstroke details, such as brush rotation, tilt, and finger pressure changes.
  2. The visualization module combines overall character comparison, rhythm observation, and detailed stroke-by-stroke analysis, helping students progressively understand techniques.
  3. The system supports both teacher-guided and self-learning scenarios, enhancing teaching efficiency while promoting student autonomy.
  4. Uniquely introduces dynamic capture technology based on low-cost devices, offering a more economical and user-friendly alternative to robotic arms or high-end equipment.

• Implementation Steps:

  1. Use smartphone cameras (top and side views) to record the writing process, inertial sensors to capture brush movement data, and pressure sensors to record finger pressure.
  2. Apply image processing algorithms to correct writing trajectories, aligning dynamic writing actions with the resulting calligraphy forms.
  3. Provide a three-layer visualization module:
     • Overall Character Comparison: Helps students understand the overall structure and composition of characters.
     • Writing Rhythm Analysis: Displays finger pressure and speed distribution, supplemented by color coding.
     • Stroke-by-Stroke Detail Analysis: Visualizes brush rotation direction and tilt angle, with curves representing speed and pressure.

Research Outcomes

• Specific Achievements:

  1. Developed a complete writing capture and visualization system, including data collection, processing, and analysis modules.
  2. The system significantly improved students' understanding of complex skills such as central brushstroke techniques (zhongfeng) and stroke textures.
  3. Students received clear dynamic feedback, with reports indicating that the system effectively reduced their reliance on teachers.

• Advantages Over Existing Solutions:

  1. Compared to simple character imitation technologies, CalliSense more comprehensively captures the dynamic writing process.
  2. The system's hardware is low-cost, and the sensors minimally impact the natural writing experience, making it widely applicable.
  3. Compared to the high cost and complexity of mechanical feedback systems, CalliSense is easier to deploy and more user-friendly.

• Experimental and Evaluation Results:

  1. Students demonstrated significant skill improvement during experiments, with enhanced understanding of five fundamental calligraphy techniques (central brushstroke, wrapped brushstroke, brush tilt, finger pressure influence, and writing speed).
  2. Classroom engagement and interest scores increased, and teaching efficiency improved significantly with the system's assistance.
  3. The NASA-TLX (Task Load Index) indicated that the system effectively reduced students' cognitive load while enhancing the intuitiveness of feedback.

• Limitations and Future Directions:

  1. The system's capture accuracy for small characters (e.g., small script) needs improvement.
  2. Support for independent learning by beginners requires further enhancement, potentially through the integration of AI models to automatically recognize and interpret stroke data.
  3. Future technological extensions include optimizing tracking algorithms for complex lighting conditions, supporting real-time feedback systems, and enabling long-term writing progress tracking to facilitate remote teaching.

Conclusion

CalliSense advances calligraphy education from static imitation to dynamic cognition, providing students with clear operational feedback and improving the efficiency of learning calligraphy techniques. It serves not only as a new tool for the preservation of intangible cultural heritage but also as a potential technological aid for other disciplines and art forms involving fine motor skills. There remains room for improvement, such as conducting longitudinal studies to evaluate long-term effects and developing calligraphy education systems based on collective data.