A Personalized and Adaptable User Interface for a Speech and Cursor Brain-Computer Interface

Paper Title

A Personalized and Adaptable User Interface for a Speech and Cursor Brain-Computer Interface

Publication Info

• Topic area: Brain-computer interface (BCI) user interface design for individuals with severe paralysis.
• Keywords: Brain-computer interface, user-centered design, assistive technology, speech decoding, cursor control, ALS, paralysis, multimodal interaction, personalization, adaptability.

Background and Problem

• Problem / challenge: Existing brain-computer interface (BCI) systems lack user-centered designs for sustained daily use, particularly for individuals with severe paralysis. Current assistive technologies face limitations such as slow typing speeds, user fatigue, and challenges with precision or reliability.
• Significance: Effective BCIs can restore independence and improve quality of life for individuals with paralysis by enabling communication and computer interaction.
• Motivation and related work: Previous research has focused on decoding algorithms and control modalities for BCIs but has largely overlooked user interface (UI) design for practical, long-term use. While non-implantable BCIs and alternative augmentative communication (AAC) systems exist, they are limited in speed, precision, and adaptability. Implantable BCIs offer higher fidelity control but require personalized and adaptable UIs to meet diverse user needs.

Solution

• Proposed approach: A personalized and adaptable UI for an intracortical BCI system, developed through iterative co-design with users, enabling multimodal interaction and real-time communication.
• Novelty:
  1. Long-term co-design process with a user (T15) to create a personalized BCI interface, demonstrating transferable design principles.
  2. Flexible system architecture with a shared backend and adaptable frontend, allowing customization for different users (e.g., T19).
  3. Integration of multimodal control methods (speech decoding, neural cursor, gestures, eye tracking) for context-dependent interaction.
  4. Iterative development of sentence correction features to balance accuracy and usability.
• Procedure and key techniques:
  1. Iterative co-design with participant T15 over 22 months, including daily use, surveys, and feedback sessions.
  2. Development of a finite state machine (FSM)-based UI with modular components for speech decoding, cursor control, and sentence correction.
  3. Deployment of the system with a second participant (T19) to demonstrate adaptability to different control schemes and needs.
  4. Evaluation through longitudinal usage data, surveys, and system logs.

Results

• Concrete findings:
  • T15 used the system for over 4000 hours across 22 months, achieving up to 60 words per minute and maintaining full-time employment.
  • Sentence correction accuracy improved from 40-41% to 59% through iterative design, with correction times increasing from 19 to 62 seconds.
  • Word-level correction was used in 91.2% of successfully corrected sentences, with manual typing yielding the highest accuracy (76%).
  • T15 rated independence while using the system at 4-5 out of 5, though setup required caregiver assistance.
• Advantage over baselines:
  • Multimodal interaction (e.g., neural cursor, eye tracking) addressed limitations of single-modality systems.
  • Personalized sentence correction features enabled higher accuracy and usability compared to initial designs.
  • Adaptable architecture allowed deployment with a second user (T19) with different needs and control methods.
• Experiments / evaluation:
  • Longitudinal study with T15, including surveys (assistive technology assessment, personal use task evaluation, user-centered design questionnaire) and system usage logs.
  • Deployment with T19, adapting the system for NATO codeword-based speech decoding and limited physical movement.
• Limitations and future work:
  • Limited user base due to clinical trial constraints and the need for surgical implantation.
  • System does not proactively adapt to user performance or context.
  • Future work should explore larger-scale testing and further automation of adaptation processes.

Summary

This study presents a personalized and adaptable UI for an intracortical BCI system, enabling individuals with severe paralysis to communicate and interact with computers independently. Through a 22-month co-design process with participant T15, the system was tailored to support multimodal interaction (speech decoding, neural cursor, eye tracking) and iterative sentence correction. Deployment with a second participant (T19) demonstrated the system’s adaptability to different needs and control methods. The findings highlight the importance of personalization, multimodal redundancy, and flexible correction options in assistive BCI design, offering a model for future user-centered development.