``Just Not Together'': The Experience of Videoconferencing for People with Aphasia during the Covid-19 Pandemic

Title of the Paper

"Just Not Together": The Experience of Videoconferencing for People with Aphasia during the Covid-19 Pandemic

Bibliographic Information

• Research Domain: Human-Computer Interaction (HCI), Remote Communication, and Assistive Technologies
• Keywords: Videoconferencing, Aphasia, Covid-19 Pandemic, Multimodal Communication, Accessibility, Online Interaction, Social Isolation, Digital Health, Identity Representation, User Interface Design

Research Background and Issues

• Problems and Challenges:

  • People with aphasia often rely on various communication methods (e.g., gestures, writing, props) to compensate for language deficits, collectively referred to as "multimodal communication."
  • The Covid-19 pandemic shifted communication from face-to-face interactions to videoconferencing, yet it remains unclear whether videoconferencing negatively impacts the communication methods and efficiency of people with aphasia.
  • Common limitations of videoconferencing (e.g., delays, lack of eye contact, difficulties with turn-taking) may pose additional challenges for aphasia patients.

• Significance of the Study:

  • Videoconferencing has become a routine mode of communication, raising urgent demands for fairness and accessibility.
  • Understanding the communication experiences of people with aphasia in videoconferencing can help improve technology design and reduce inequalities in social communication.

• Motivation and Related Work:

  • Existing research on aphasia and technology use, as well as language rehabilitation, indicates that technology can support people with aphasia, but there is limited research on the specific impacts of videoconferencing.
  • Studies in other domains, such as musical performance and telemedicine, have revealed certain challenges of remote communication, but the specific needs of aphasia patients remain underexplored.
  • This study aims to fill this gap by focusing on the communication experiences of people with aphasia in videoconferencing, particularly the applicability of multimodal communication.

Approach

• Research Methods:

  1. Observational Study: Recording and analyzing videoconferencing sessions of online communication groups for people with aphasia.
  2. Interview Study:
     • Interviews with speech therapists and volunteers to explore their professional experiences and observations of videoconferencing.
     • Interviews with people with aphasia to capture their personal experiences and communication details.
  3. Comprehensive Analysis: Extracting key issues and impacts through thematic analysis.

• Innovative Contributions:

  • This study examines videoconferencing platforms from the perspective of "multimodal communication," uncovering unique communication strategies and barriers.
  • It provides a series of recommendations and directions for improving videoconferencing, including technical features, user interface design, and patterns of social interaction.

Findings

Observational Results

• Communication Strategies:

  • The use of props is the most common communication behavior, demonstrating the improvisational creativity of home environments (e.g., showing photos, writing words).
  • Gestures (e.g., numeric hand signs, pointing actions) are also frequent communication methods.
  • Challenges in the videoconferencing environment include issues with the positioning of props and gestures on the screen, mirroring effects, and unintended obstructions.

• Errors and Repairs:

  • Delays in videoconferencing lead to overlapping speech and difficulties in turn-taking, increasing the cost of interaction repair.
  • Technical issues (e.g., video freezing, disconnections) disrupt communication fluency.

Challenges

• Lack of Body Language: Videoconferencing screens fail to display full body language, affecting subtle aspects of communication.
• Formalization and Lack of Intimacy: Communication becomes more structured in videoconferencing, lacking the natural informal interactions and emotional connections of face-to-face exchanges.
• Household Distractions: The uncontrollable home environment and involvement of third parties (e.g., family members) increase dependency and affect privacy for people with aphasia.
• Interface Complexity: Inconsistent user interfaces across devices pose additional challenges for aphasia patients.

Benefits

• Advantages of Props: Using household props enriches communication, with personalized items or displayed content providing new opportunities for social dialogue.
• Convenience: Geographical constraints are minimized, saving travel costs.
• Self-Monitoring: Video windows help patients observe their expressions and adjust their communication behaviors.

Experimental Evaluation

• Strengths: Videoconferencing served as a lifeline for maintaining social connections during the pandemic, addressing basic communication needs for people with aphasia.
• Limitations: Lack of technological support for nonverbal communication; insufficient academic datasets supporting aphasia users.

Future Directions

• Technical Improvements:

  • Provide clearer visual feedback (e.g., intelligent mirroring, gesture recognition).
  • Develop simplified interactive interfaces to reduce complexity.
  • Build multimodal feedback mechanisms to support diverse communication methods.

• Platform Optimization:

  • Enhance features for recognizing aphasia-specific needs, such as adapting speech models.
  • Implement flexible user view management functions (e.g., prioritizing nonverbal behaviors).
  • Design interactive features to track gestures or prop usage.

• Behavioral and Social Parameters:

  • Explore ways to create more informal interaction scenarios in videoconferencing.
  • Strengthen conversation control and turn-taking capabilities to improve equal participation.

Conclusion

• Videoconferencing platforms require further improvements to support the communication needs of people with aphasia, particularly in adapting to multimodal communication strategies.
• The findings of this study lay the foundation for designing more inclusive videoconferencing platforms, while emphasizing the necessity of technological and social support for people with aphasia.