AmIWrite: Exploring Scalable One-on-One Handwriting-Based Tutoring for Mathematical Problem-Solving with an LLM-Powered AI Tutor

Paper Title

AmIWrite: Exploring Scalable One-on-One Handwriting-Based Tutoring for Mathematical Problem-Solving with an LLM-Powered AI Tutor

Publication Info

• Topic area: Scalable AI-driven handwriting-based tutoring for STEM education.
• Keywords: Handwriting-based tutoring, LLM-powered tutor, STEM education, mathematical problem-solving, AI tutoring systems, multimodal interaction, co-speech handwriting, linear algebra, personalized learning, educational technology.

Background and Problem

• Problem / challenge: Traditional handwriting-based STEM tutoring is highly effective but resource-intensive, making it difficult to scale. Current online platforms fail to replicate the dynamic, real-time, and personalized nature of handwriting-based tutoring.
• Significance: Scaling handwriting-based tutoring could democratize access to high-quality STEM education, addressing inequities in learning opportunities and improving conceptual understanding in critical fields like mathematics.
• Motivation and related work: Prior efforts in online education (e.g., digital whiteboards, AI-powered systems like Photomath) lack real-time, adaptive handwriting interactions. Large Language Models (LLMs) have shown promise in personalized tutoring but have not been applied to synchronized handwriting-based instruction. This paper addresses the gap by leveraging LLMs for scalable, multimodal tutoring.

Solution

• Proposed approach: AmIWrite, an LLM-powered AI tutoring system that provides real-time co-speech handwriting interactions on tablet devices, simulating one-on-one tutoring for mathematical problem-solving.
• Novelty:
  1. Introduction of a design space for handwriting interactions (e.g., writing, annotations) tailored for STEM tutoring.
  2. Development of AmIWrite, a system that synchronizes verbal explanations with in-situ handwritten feedback on a shared canvas.
  3. Evaluation of AmIWrite’s usability and effectiveness through a controlled user study comparing it to a text-based AI tutor.
• Procedure and key techniques:
  • The system processes student handwriting and voice input, integrates course materials, and generates multimodal feedback (e.g., annotations, verbal explanations).
  • Handwriting interactions include declarative, procedural, and selective writing, as well as annotations like underlines, circles, arrows, and correctness marks.
  • The system follows a Gradual Release of Responsibility model: lecture, guided practice, and independent practice.
  • A user study evaluates learning outcomes, usability, and workload compared to a text-based baseline.

Results

• Concrete findings:
  • Learning gains were comparable between AmIWrite and the baseline system (e.g., multiplication task: AmIWrite mean = 0.731, baseline mean = 0.626).
  • AmIWrite significantly improved user experience metrics, including ease of understanding (mean = 6.45 vs. 4.97, p < 0.001) and reduced cognitive load (NASA-TLX mental demand: mean = 29.50 vs. 43.12, p < 0.01).
  • High system usability score for AmIWrite (SUS mean = 78.38 vs. baseline mean = 63.44, p < 0.001).
• Advantage over baselines:
  • AmIWrite outperformed the baseline in user experience, reducing attention-switching demands, improving error localization, and providing more natural and engaging interactions.
  • Participants preferred AmIWrite’s synchronized verbal and visual feedback over the text-based feedback of the baseline.
• Experiments / evaluation:
  • A within-subjects study with 40 participants compared AmIWrite to a ChatGPT-style text-based tutor on two linear algebra topics (matrix rank and matrix multiplication).
  • Evaluation metrics included pre- and post-test learning gains, user experience questionnaires, NASA-TLX workload, and System Usability Scale (SUS) scores.
• Limitations and future work:
  • Current implementation focuses on structured handwriting in linear algebra; unstructured handwriting and domain-specific diagrams remain challenging.
  • Localization errors and occasional hallucinations in feedback were observed.
  • Response latency (∼15 seconds) was noted as a limitation.
  • Future work includes improving diagram recognition, reducing response time, and incorporating multimodal inputs (e.g., facial expressions, voice tone) for adaptive personalization.

Summary

AmIWrite is an LLM-powered AI tutoring system designed to replicate one-on-one handwriting-based STEM tutoring at scale. By integrating real-time co-speech handwriting interactions with verbal explanations, AmIWrite enhances the learning experience while maintaining the pedagogical benefits of traditional tutoring. A user study demonstrated that AmIWrite significantly improves user experience and reduces cognitive load compared to a text-based baseline, though learning gains were similar. Future work will address limitations in handling unstructured handwriting, reducing response latency, and enhancing personalization to broaden the system’s applicability across STEM domains.