Do-It-Yourself AAC: Co-Designing User-Programmable AI Communication Tools with People with Aphasia

Paper Title

Do-It-Yourself AAC: Co-Designing User-Programmable AI Communication Tools with People with Aphasia

Publication Info

• Topic area: Accessible AI-driven communication tools for people with aphasia
• Keywords: Aphasia, AAC, generative AI, end-user programming, co-design, accessibility, visual programming, tangible interfaces, participatory design, communication technology

Background and Problem

• Problem / challenge: Existing AAC tools for people with aphasia (PWA) are difficult to customize and often fail to meet diverse personal communication needs. Generative AI offers potential for personalization, but current systems are not accessible or adaptable for PWA.
• Significance: Addressing this gap can empower PWA to create personalized communication tools, improving their conversational abilities and quality of life.
• Motivation and related work: Previous research has explored AAC technologies, generative AI for communication, and end-user programming, but there is little investigation into how these can be made accessible for PWA. This paper seeks to bridge this gap by enabling PWA to design their own AI-driven communication tools through tangible programming.

Solution

• Proposed approach: A visual user-programming method using tangible block-based programming to allow PWA to create personalized AI-driven communication tools.
• Novelty:
  1. Introduced tangible programming blocks as an accessible medium for PWA to design AI-enhanced communication tools.
  2. Conducted co-design workshops to explore how PWA envision and create custom tools for their communication needs.
  3. Demonstrated how visual programming can act as a scaffold for articulating technology requirements in participatory design.
• Procedure and key techniques:
  • Conducted semi-structured interviews with eight PWA to understand their communication needs and perspectives on generative AI.
  • Held in-person co-design workshops where participants used physical code blocks to create hypothetical AI programs tailored to their real-life communication scenarios.
  • Analyzed participant-created programs and feedback using thematic analysis.

Results

• Concrete findings:
  • All participants successfully created hypothetical programs using physical code blocks, addressing scenarios like doctor visits, grocery shopping, and teaching.
  • Participants valued AI functions like sentence suggestion and grammar correction but had varied preferences for image-based features.
  • Tangible programming blocks reduced language barriers and enabled PWA to articulate their needs effectively.
• Advantage over baselines:
  • Unlike rigid AAC tools, the proposed method allowed for personalization and flexibility, accommodating the diverse needs of PWA.
  • Tangible programming provided an accessible alternative to language-intensive design methods.
• Experiments / evaluation:
  • Participants included eight PWA with mild to moderate aphasia, recruited from local support groups.
  • Two-part study: virtual interviews (session one) and in-person co-design workshops (session two).
  • Evaluation focused on participants’ ability to create programs, their perspectives on AI features, and the accessibility of the programming method.
• Limitations and future work:
  • Limited sample size and focus on mild-to-moderate aphasia profiles.
  • Future work should refine the programming framework, include a broader range of aphasia profiles, and explore real-world deployment of user-programmable AAC systems.

Summary

This study introduces a tangible block-based programming method to enable people with aphasia (PWA) to create personalized AI-driven communication tools. Through semi-structured interviews and co-design workshops with eight participants, the research highlights how visual programming can act as an accessible scaffold for articulating technology needs. Participants successfully created hypothetical programs tailored to their communication challenges, demonstrating the potential of user-programmable AAC to enhance personalization and agency. The findings underscore the importance of accessibility in participatory design and suggest that tangible programming can make AI-enhanced communication tools more adaptable for PWA. Future work should expand the framework and investigate its deployment in real-world AAC systems.