Breaking Badge: Augmenting Communication with Wearable AAC Smartbadges and Displays

Document Title

Breaking Badge: Augmenting Communication with Wearable AAC Smartbadges and Displays

Document Information

• Subject Area: Assistive Hearing Technology and Augmentative/Alternative Communication (AAC) Technology Design
• Keywords: AAC, Augmentative and Alternative Communication, Accessibility Technology, Wearable Devices, Smart Badges, Invisible Disabilities

Research Background and Issues

• What problems or challenges did the authors identify?

  1. Current augmentative and alternative communication (AAC) technologies often fail to enhance users' existing communication strengths, instead restricting intuitive communication pathways such as natural voice and nonverbal communication (e.g., gestures, facial expressions).
  2. AAC devices are frequently criticized for slow communication speeds, limited autonomy, high learning demands, and bulky designs that may cause social stigma and lead to device abandonment.
  3. Many individuals live with complex communication needs (e.g., aphasia caused by stroke) and often face highly stressful public communication scenarios.

• Why is this issue important? The proportion of individuals with complex communication needs (e.g., aphasia) increases with age and the global trend of population aging. Better-designed assistive technologies are crucial for improving the quality of daily life for these populations.

• Research Motivation and Related Work The authors referenced prior studies, such as the TalkAbout application designed for aphasia patients, and early explorations of assistive technologies for invisible disabilities (e.g., badges). They argue that current AAC devices fail to effectively meet complex communication needs and draw inspiration from low-tech AAC research to advocate for streamlined, personalized, and wearable solutions.

Solution

Methods/Solutions

The authors designed and evaluated two wearable AAC device prototypes:

1. InkTalker Smart Badge:
   • Utilizes a low-power electronic ink (eInk) screen to display customizable messages, serving as a tool to reveal invisible disabilities and aid conversations.
   • Supports multifunctional scenarios: requesting help, conveying information about aphasia, displaying interests, etc.
   • Flexible wearable formats: can be attached to a shirt pocket, lanyard, or backpack.
2. WalkieTalkie Application:
   • An iOS mobile application that transforms a smartphone into a feature-rich public display.
   • Offers customizable badges, multimodal interaction, image classification (e.g., photo-assisted communication), search tools, and more.
   • Multiple wearing options, including lanyards, belts, or transparent phone pouches.

Innovations

• Low learning curve: Both devices focus on simplifying user operations and reducing cognitive load.
• Support for multimodal communication: Enhances users' existing communication abilities by supporting gestures and nonverbal interactions in addition to speech generation.
• Sociocultural sensitivity: Devices are small, lightweight, and can be concealed or integrated with other low-tech devices used in daily life, reducing social stigma.

Implementation Steps

1. Collaboration and Needs Assessment:
   • Conducted in-depth needs research with 19 aphasia patients, including identifying challenging communication scenarios and creating prototype designs.
   • Employed accessible co-design methods such as SWIM scenario design workshops and low-fidelity concept modeling.
2. Device Prototype Development:
   • Developed the InkTalker using Raspberry Pi Pico-W hardware, leveraging an eInk screen for low-power communication.
   • Created the WalkieTalkie app, fully utilizing smartphone hardware to enhance communication.
3. Experimentation and Evaluation:
   • Analyzed the performance and user feedback of the two devices through focus groups and experiential prototype testing.

Research Outcomes

Specific Outcomes

1. Technical Effectiveness:
   • Both devices were proven effective in assisting users in high-pressure communication scenarios, acting as a "safety net" to prevent communication breakdowns.
   • Devices supported users' multimodal communication abilities (e.g., language, gestures, nonverbal interactions), significantly improving independence and communication confidence.
2. User Feedback:
   • Users expressed positive feedback on the low-power, modular design and customizable content of the wearable smart badge.
   • The WalkieTalkie app was recognized for its potential in search assistance, image classification, and symbol-based communication support.

Advantages

• Compared to traditional AAC devices, InkTalker and WalkieTalkie are lighter, more portable, and require less learning effort.
• They do not interfere with users' intuitive communication pathways, encouraging natural interactions while enhancing sociocultural adaptability.
• The devices allow flexible content customization, supporting personalized needs across different contexts and physical conditions.

Experimental or Evaluation Results

• In focus groups, most users strongly agreed that the two devices were suitable for supporting their communication needs.
• Although there were minor areas for improvement, such as small button design and image classification accuracy, overall feedback indicated the devices demonstrated strong practicality in supporting communication.

Limitations and Future Directions

• Limitations:
  • The study sample was relatively small and primarily focused on individuals with aphasia, requiring further validation for other groups with complex communication needs.
  • Lacked long-term deployment data in real-world high-pressure communication environments.
• Future Directions:
  • Optimize device design through real-world usage scenarios and explore broader configurations.
  • Consider adopting new technologies to address challenges such as underrepresentation of minority groups and cross-cultural adaptability.