Interactive AR applications for Nonspeaking Autistic People? A Usability Study

Title of the Paper

Interactive AR Applications for Nonspeaking Autistic People? - A Usability Study

Paper Information

• Field of Study: Testing the usability of augmented reality (AR) technology for nonspeaking autistic individuals
• Keywords: nonspeaking autism, augmented reality, usability study, assistive technology, HoloLens

Research Background and Issues

• Identified Problems or Challenges:
  • Approximately one-third of autistic individuals are unable to communicate effectively through speech, yet most lack access to effective alternative communication methods.
  • Nonspeaking autistic individuals often face challenges in motor planning and control, as well as severe sensory issues, making it difficult for them to participate in conventional learning or social activities.
  • Current research on AR technology for autism largely focuses on individuals who can communicate verbally, with limited studies on nonspeaking individuals.
• Significance:
  • AR technology can provide structured learning opportunities for nonspeaking autistic individuals, particularly in developing skills related to alternative communication methods such as typing.
  • Assessing whether nonspeaking individuals can tolerate head-mounted AR devices and interact with virtual buttons is a crucial prerequisite for further technological development.
• Research Motivation and Related Work:
  • Existing literature suggests that AR can support learning and social skills for autistic individuals through customized interactive methods.
  • Technologies used in similar studies (e.g., Google Glass) have not validated the feasibility of nonspeaking individuals using modern AR devices like HoloLens 2.
  • Exploring the ability of this target group to tolerate and operate virtual objects is necessary to address the research gap.

Solution

• Proposed Solution:
  • Developed a virtual button game running on the HoloLens 2 device, featuring progressively challenging button-pressing tasks.
  • The game provides an opportunity for nonspeaking individuals to adapt to head-mounted devices through guided interactions with virtual buttons of varying sizes and quantities.
• Innovations:
  • The first usability study designed specifically for nonspeaking autistic individuals using HoloLens 2 for interaction.
  • The game incorporates a step-by-step task design informed by input from the user community (families, professionals, autistic individuals).
  • Offers a combination of untimed tasks and timed challenges, using visual changes and sound effects as feedback mechanisms to enhance user engagement.
• Implementation Steps and Key Technologies:
  1. Utilized HoloLens 2's hand-tracking technology to allow users to interact directly with virtual buttons using their fingers.
  2. Designed the game with three rounds of tasks, progressively increasing the number of buttons and decreasing their size. Each round includes untimed practice tasks and one or two timed challenges.
  3. Provided visual text and voice guidance, with animations as rewards after each button interaction.
  4. Data collection included device tolerance duration, user input response times, and game completion levels.

Research Outcomes

• Specific Findings:
  • Among 17 nonspeaking autistic participants, 88% were able to wear the device and start the game.
  • Of the 15 participants who engaged with the game, 11 completed all tasks, demonstrating significant improvement in learning abilities.
  • Data indicated that most users could enhance their button-clicking efficiency through interaction, despite increasing game difficulty across rounds.
• Advantages Over Existing Solutions:
  • Compared to static traditional support methods, AR enhances interactivity, providing a foundation for practicing complex skills such as typing.
  • Offers insights into missed interactions and behavioral data, revealing technological limitations (e.g., button sensitivity) and user characteristics (e.g., rapid adaptation).
• Experimental or Evaluation Results:
  • Device Tolerance: 67% of users completed the experiment without interruptions, while others also showed high adaptability.
  • Learning Ability Improvement: The majority of users progressively improved their operational skills through the task rounds.
  • Analysis of button-clicking errors indicated that most issues could be addressed by adjusting the user interface and interaction settings.
• Limitations and Future Directions:
  • Limitations: Small sample size, significant gender imbalance, restricted to adolescent and young adult groups, and limited to HoloLens 2 devices.
  • Future Directions:
    1. Develop AR-based multisensory interaction modules to support complex skills.
    2. Explore the potential of other AR devices or accessories (e.g., haptic feedback).
    3. Extend research to broader age groups, particularly children and older adults.