Lost in Instructions: Study of Blind Users’ Experiences with DIY Manuals and AI-Rewritten Instructions for Assembly, Operation, and Troubleshooting of Tangible Products

Paper Title

Lost in Instructions: Study of Blind Users’ Experiences with DIY Manuals and AI-Rewritten Instructions for Assembly, Operation, and Troubleshooting of Tangible Products

Publication Info

• Topic area: Accessibility and assistive technologies for blind users in DIY tasks.
• Keywords: Blind users, DIY tasks, AI tools, product manuals, accessibility, spatial reasoning, troubleshooting, assembly, non-visual guidance, assistive technologies.

Background and Problem

• Problem / challenge: Blind users face significant challenges accessing and interpreting product manuals for DIY tasks due to their visual-centric nature. AI tools, while promising, often fail to provide actionable non-visual guidance, exacerbating these challenges.
• Significance: Effective solutions are critical for enabling blind users to independently assemble, operate, and troubleshoot tangible products, fostering autonomy and reducing reliance on sighted assistance.
• Motivation and related work: Prior research has explored AI tools for navigation, cooking, and general information access but has not adequately addressed their role in product-manual-mediated DIY tasks. Existing tools often rely on visual references and fail to provide tactile, spatial, and orientation cues necessary for blind users.

Solution

• Proposed approach: Systematic investigation into blind users’ experiences with product manuals and AI tools through interviews and usability studies, identifying gaps and proposing design improvements.
• Novelty:
  1. Empirical study of blind users’ interactions with AI tools and product manuals for DIY tasks.
  2. Identification of specific challenges, including vision-default bias, inconsistent guidance, and hallucinations in AI-generated instructions.
  3. Proposal of structured, layered instruction templates and design recommendations for both product manuals and AI tools.
• Procedure and key techniques:
  • Conducted semi-structured interviews with 15 blind participants to capture lived experiences and challenges.
  • Performed usability studies with 7 participants, observing real-time DIY task execution using AI tools and manuals.
  • Analyzed data using thematic coding and metrics such as task completion rate, accuracy, and workload (NASA-TLX).

Results

• Concrete findings:
  • Task completion rate was 2.4%, with step accuracy below 50% across all tasks.
  • AI tools frequently defaulted to vision-centric guidance, omitted tactile and spatial cues, and hallucinated steps or components in 64% of responses.
  • Participants relied on fragmented sources (manuals, AI tools, tutorials) but succeeded independently only 10% of the time.
• Advantage over baselines:
  • Structured, layered instructions reduced clarification needs by 80% compared to AI-generated guidance.
  • Recommendations for product manuals and AI tools promise improved accessibility and independence.
• Experiments / evaluation:
  • Tasks included assembly, learning to operate, and troubleshooting products using paper, PDF, and web manuals.
  • Metrics included task completion rate, step accuracy, human intervention counts, and perceived workload (NASA-TLX).
• Limitations and future work:
  • Small sample size; findings may under-represent novice or non-English-speaking users.
  • Focused on current AI tools; future models may alter observed patterns.
  • Lab-based study may not fully capture real-world dynamics.

Summary

This study highlights the persistent challenges blind users face with product-manual-mediated DIY tasks, including reliance on visual references, inconsistent AI guidance, and fragmented support sources. Despite their importance, manuals and AI tools often fail to provide actionable non-visual instructions, forcing users to rely on workarounds like tutorials or sighted assistance. The findings inform design recommendations for product manufacturers and AI developers, emphasizing structured, stepwise guidance with tactile, spatial, and orientation cues to support more accurate and independent task completion. Future work should expand participant diversity, test broader product categories, and prototype systems implementing these recommendations.