Designing for Harm Reduction: Communication Repair for Multicultural Users' Voice Interactions

Title of the Paper

Designing for Harm Reduction: Communication Repair for Multicultural Users’ Voice Interactions

Paper Information

• Research Area: Human-Computer Interaction and Multicultural Technology Design
• Keywords: Language Technology, Voice Assistants, Automatic Speech Recognition, Communication Repair, Conversational User Interfaces, Multilingual, Multicultural, Harm, Harm Reduction

Research Background and Problem

• Identified Problems or Challenges by the Authors

  • The quality of voice assistant services is insufficient, particularly in supporting people of color and non-native English speakers. Existing research has primarily focused on service quality issues, while the downstream harms caused by these issues have not been systematically explored.
  • Users have reported functional disparities and negative emotional impacts caused by speech recognition errors, such as code-switching (adjusting accent or language to make the assistant understand commands) and the inability to recognize culturally significant names.
  • How the frustrations of multicultural users in voice interactions materialize and affect their lives, as well as what designs can help users recover from these impacts, remain underexplored.

• Why This Problem is Important

  • With the rapid adoption of voice assistants in modern society, their influence continues to expand. If certain user groups consistently fail to benefit from such technologies, it not only challenges technological fairness but also risks exacerbating social inequalities.
  • Repairing communication errors is not only about improving system functionality but also about mitigating the psychological and social negative impacts users experience due to interaction failures.

• Research Motivation and Related Work

  • Systematically exploring multicultural users’ perceptions of voice assistants and the specific harms they experience by combining psychological insights with human-computer interaction design.
  • Developing conversational repair strategies that effectively reduce harm, including identity affirmation, cultural sensitivity, and responsibility shifting.

Proposed Solution

• Proposed Solution

  • Create a taxonomy of pain points and harms experienced by multicultural users.
  • Design psychology-based error repair strategies: identity affirmation, cultural sensitivity, and responsibility shifting.
  • Provide design guidelines for voice technology to reduce harm experienced by multicultural users during interactions with voice assistants.

• Innovative Contributions

  • The first application of a socio-technical harm taxonomy to address social inequality issues in voice assistants.
  • Development of communication repair design strategies tailored specifically for multicultural users.
  • Design recommendations based on emotional and visual research findings from participant studies, showcasing innovation in participatory research methods.

• Implementation Steps and Techniques

  • Collect data through interviews and design activities, involving the experiential stories and interaction design tasks of 16 participants.
  • Analyze data using open coding and existing harm taxonomies, and propose design strategies based on psychological principles.
  • Design directions include optimizing error message content (e.g., affirming identity, leveraging cultural knowledge, taking responsibility for errors), adjusting message frequency and timing, and avoiding intrusive cultural references.

Research Outcomes

• Specific Outcomes

  • Proposed a detailed taxonomy of harms experienced by multicultural users, including relational harm, loss of service or benefits, increased labor, identity and cultural harm, physical danger, and emotional harm.
  • Developed psychology-based communication repair strategies with a particular emphasis on cultural sensitivity and responsibility shifting.
  • Provided design guidelines for voice technology, such as using intermittent identity affirmation to mitigate negative impacts and respecting the diverse needs of multicultural users.

• Advantages Over Existing Solutions

  • Complements existing discussions on functional optimization with a focus on harm repair from a psychological perspective.
  • Emphasizes meeting the needs of marginalized users in design while providing universal benefits for mainstream users.
  • Supports generative practices for improving error recovery experiences in voice assistants rather than merely documenting problems.

• Experimental or Evaluation Results

  • Verified the potential of communication repair designs through design activities with 16 multicultural participants, revealing users’ conceptualizations of voice assistants.
  • Found that users were uncomfortable with overly personalized or culturally specific error messages, and highlighted that short and concise repair messages better meet user needs.

• Limitations and Future Directions

  • Future research should focus on testing the functionality and user experience effects of the proposed design guidelines.
  • Explore the application of design principles to error repair in other language technologies or artificial intelligence domains.
  • Further expand the research sample to capture the needs of more marginalized groups (e.g., cross-cultural, non-native speakers, or individuals from specific socioeconomic backgrounds).

Conclusion

This study provides an innovative framework for addressing the harms experienced by multicultural users of voice assistant technologies. By employing psychology-driven communication repair strategies, it helps mitigate the negative impacts of misrecognition. The findings contribute to promoting fairness in language technologies and offer valuable design directions, serving as an important reference for the fields of human-computer interaction and socio-technical harm research.