"Voices Help Correlate Signs and Words": Analyzing Deaf and Hard-of-Hearing (DHH) TikTokers’ Content, Practices, and Pitfalls

Document Title

"Voices Help Correlate Signs and Words": Analyzing Deaf and Hard-of-Hearing (DHH) TikTokers’ Content, Practices, and Pitfalls

Document Information

• Subject Area: Human-Computer Interaction, Social Media, and Accessible Video Content Creation
• Keywords: DHH (Deaf and Hard-of-Hearing), TikTok, content creation, multimodal, sign language, captions, audio features, accessibility design, social media, audience engagement

Research Background and Issues

• Identified Problems and Challenges:

  1. Although video-sharing platforms like TikTok provide new opportunities for Deaf and Hard-of-Hearing individuals to create content using sign language, the challenges of multimodal creation practices (including captions, audio, etc.) have not been fully explored.
  2. Most existing studies focus on the experiences of Deaf and Hard-of-Hearing individuals as "content consumers" rather than "content creators," while neglecting the reactions of general audiences to DHH content.
  3. Sign language content creation can enhance communication between Deaf and hearing communities, but issues such as missing captions and audio-video desynchronization often hinder content creation.

• Importance of the Research:

  1. Content creation enables Deaf and Hard-of-Hearing individuals to showcase their cultural identity in public while fostering inclusive social environments.
  2. Systematic analysis and exploration of multimodal content creation practices and audience reactions can provide valuable insights for future video platform design and DHH creators’ practices.

• Research Motivation:

  1. Investigate the content creation practices of Deaf and Hard-of-Hearing individuals on TikTok, including content themes, creation methods, and key multimodal features, to better understand the actual needs of the DHH community.
  2. Examine the impact of design elements such as sign language, captions, and audio features on audience engagement (e.g., likes, shares, and views).

Solutions

• Method Overview:

  1. Using a mixed-methods approach, the study systematically analyzed 308 TikTok videos, focusing on their themes, creation practices, potential pitfalls, and audience reactions.
  2. Conducted qualitative analysis of video themes, practices, and comments, alongside statistical modeling (negative binomial regression) to quantify the impact of practices on viewership metrics (likes, views, etc.).

• Innovative Contributions:

  1. For the first time, comprehensively analyzed the multimodal practices of DHH creators and audience responses to their content, covering captions, audio, background music, sign language, and other features.
  2. Highlighted the importance of improving DHH creators’ audio features (e.g., voiceovers and dubbing), which have been largely overlooked in existing research.
  3. Explored the dual effects of disclosing Deaf identity on authenticity and audience trust.

Research Findings

Video Themes:

The study identified five major themes among DHH TikTok creators:

1. Disclosing Disability Challenges (26%): Includes experiences of online harassment, stereotypes, discrimination, and awkward situations in real life.
2. Sign Language Content (24%): Sign language tutorials, misuse of sign language, sign language challenges, and reaction videos.
3. Raising Awareness of Hearing Impairment (21.4%): Showcasing hearing aids, advocating for accessibility rights, and responding to comments from hearing audiences.
4. Entertainment Content (20.1%): Lighthearted and humorous videos (e.g., outfit showcases, creative skits).
5. Interactions with Social Relationships (10.4%): Depicting complex interactions between Deaf individuals and various social groups (family, friends, etc.).

Creation Practices and Pitfalls:

1. Creation Practices:

   • Captions: The majority of videos (86.04%) included captions.
   • Audio Features: Only 38.31% of videos contained synchronized voiceovers, and the combined proportion of synchronized voiceovers and dubbing was even lower (34.74% lacked any audio features).
   • Sign Language: 63.96% of videos included sign language.
   • Background Music and Lip-Syncing: Background music was moderately present (44.48%), while lip-syncing videos were less frequently included (44.16%).

2. Common Pitfalls:

   • Caption Issues (51.61% of videos with pitfalls): Missing or low-quality captions.
   • Audio-Visual Mismatch (32.26%): Desynchronization between audio and visuals.
   • Audio Quality Problems (29.03%): Issues such as fluctuating volume or unnecessary silence.

Audience Engagement and Reactions:

1. Quantitative Analysis of Audience Engagement:

   • Theme Correlation: Content centered on sign language (e.g., tutorials and challenges) was more likely to be shared; in contrast, videos under the "raising awareness of hearing impairment" theme had lower engagement (likes and shares).
   • Practice Correlation:
     • The use of sign language significantly boosted likes, views, and shares.
     • External voiceovers, background music, and lip-syncing notably reduced viewing experiences.

2. Qualitative Findings:

   • Audiences showed strong support and empathy for videos disclosing disability challenges or teaching sign language.
   • Hearing audiences expressed curiosity about the technical details of audio and background music used by DHH creators.

Limitations and Future Research Directions

• Limitations:

  1. Did not directly interview Deaf creators, leaving the specific reasons behind certain creative tendencies (e.g., choices regarding audio features) unclear.
  2. The sample may predominantly represent sign language users within the Deaf community, potentially overlooking the needs of a broader range of non-sign language Deaf individuals.

• Future Directions:

  1. Optimize voiceover technology tools specifically designed for the Deaf community.
  2. Conduct quantitative studies on the dynamic relationship between Deaf creators’ disclosure of disability identity and audience interactions (e.g., malicious comments).
  3. Explore the design and practical effectiveness of synchronized voice tools in complex contexts (e.g., coexistence of ASL and spoken language).

Practical Recommendations

1. Recommendations for TikTok Platform and Tool Design:

   • Provide automatic subtitle tools to detect audio-visual mismatches.
   • Implement real-time audio editing features to flag unstable volume levels.
   • Introduce DHH identity verification mechanisms to enhance creator credibility.

2. Recommendations for Deaf Creators:

   • Optimize audio features (e.g., synchronized voiceovers) to attract hearing audiences, especially in non-educational contexts.
   • For content raising awareness of hearing impairment, incorporate elements of sign language tutorials or entertainment to increase popularity.

3. Recommendations for the Research Community:

   • Investigate the multimodal complexities of DHH content creation in greater depth.
   • Systematically analyze the cross-platform interactions and influence of Deaf creators.

This document contributes to understanding the content creation practices, multimodal features, and audience reactions of Deaf creators, offering valuable insights for improving accessibility and inclusivity in social media platforms.