The RepairBot Framework: Touch-Aware Conversational Agent for Hands on Clothes Repair

Paper Title

The RepairBot Framework: Touch-Aware Conversational Agent for Hands-on Clothes Repair

Publication Info

• Topic area: Human-computer interaction and embodied learning for sustainable practices
• Keywords: Conversational agents, Human-Touch-Awareness, clothes repair, embodied learning, tactile feedback, repair education, sustainability, multimodal tutorials, Wizard-of-Oz, circular economy

Background and Problem

• Problem / challenge: Novices face procedural, embodied, and emotional barriers when learning to repair clothes, including limited access to interactive, context-aware teaching tools. Existing digital resources like video tutorials are passive, non-interactive, and fail to convey tactile knowledge essential for repair.
• Significance: Repairing clothes promotes sustainability by extending garment lifespans and reducing waste, but the decline in mending skills and lack of accessible teaching resources hinder broader adoption of repair practices.
• Motivation and related work: Prior research highlights the need for interactive, holistic support for repair learning. While some systems offer procedural guidance or post-hoc feedback, they lack the ability to sense and respond to users’ embodied actions, a critical gap for hands-on crafts like sewing.

Solution

• Proposed approach: RepairBot Conversation Framework (RBCF), a modular conversational agent that integrates Human-Touch-Awareness and multimodal learning resources to provide holistic, tutor-like support for novices learning clothes repair.
• Novelty:
  1. Introduction of Human-Touch-Awareness to scaffold tactile skill development and emotional regulation.
  2. Modular framework combining structured learning modules and project-based repair guidance.
  3. Integration of multimodal tutorials (video and 3D) to address spatial and procedural challenges.
  4. Use of Wizard-of-Oz (WoZ) methodology to simulate touch-aware feedback and explore its pedagogical value.
• Procedure and key techniques:
  • Conducted formative studies (autoethnography, pilot study) to identify novice challenges and design goals.
  • Developed RBCF with six repair steps, dual learning/project modules, and nine pedagogical mechanisms.
  • Implemented a hybrid chatbot architecture combining rule-based dialogue management with LLMs and WoZ touch-awareness.
  • Evaluated the system through lab (N = 16) and home (N = 9) studies, analyzing qualitative data from interviews, diaries, and chatbot logs.

Results

• Concrete findings:
  • Emotional support: RepairBot created a non-judgmental "safe space," reducing performance anxiety and fostering confidence.
  • Procedural learning: Guided steps revealed hidden garment issues and improved holistic understanding of repair processes.
  • Tactile skill development: Human-Touch-Aware feedback prompted somatic reflection, tactile experimentation, and mindful embodiment.
  • Transferable skills: Participants applied repair knowledge to shopping and wardrobe management, promoting mindful consumption.
• Advantage over baselines:
  • Structured guidance outperformed open-ended tools by lowering cognitive barriers and tailoring advice to specific garments.
  • Human-Touch-Aware feedback shifted focus from motor correction to fostering sensory investigation and emotional regulation.
  • Multimodal tutorials addressed spatial challenges better than traditional 2D media.
• Experiments / evaluation:
  • Lab study: Observed chatbot-guided repair sessions with WoZ touch-awareness, capturing real-time interactions and feedback.
  • Home study: Explored RepairBot’s use in real-world contexts, documenting autonomous repair practices and user reflections.
  • Data sources: 75 hours of video, 39 interview transcripts, chatbot logs, and 22 repair diaries analyzed via thematic analysis.
• Limitations and future work:
  • WoZ setup simulated touch-awareness, requiring future development of autonomous tactile sensing systems.
  • Limited scope focused on foundational skills rather than creative repair processes.
  • Future directions include richer multimodal datasets, integration of additional sensing modalities, and exploration of skill transfer across broader clothing management contexts.

Summary

The RepairBot Conversation Framework introduces a touch-aware conversational agent to support novices in learning clothes repair. By addressing procedural, emotional, and embodied challenges, the system fosters holistic learning through structured guidance, Human-Touch-Aware feedback, and multimodal tutorials. Evaluations demonstrated its effectiveness in building confidence, tactile skills, and transferable knowledge for sustainable fashion practices. Future work will focus on advancing autonomous touch-awareness and expanding the framework’s scope to creative and broader applications in the circular economy.