Voicemoji: Emoji Entry Using Voice for Visually Impaired People

Title of the Paper

Voicemoji: Emoji Entry Using Voice for Visually Impaired People

Paper Information

• Field of Study: Human-Computer Interaction and Accessibility Technology
• Keywords: Voice input, emoji recommendation, voice user interface, accessibility, visually impaired users

Research Background and Problem Statement

• Problems and Challenges:
  • Visually impaired users need to browse emoji lists item by item using screen readers, which is a slow and time-consuming process.
  • Current emoji input methods involve cognitive burden, excessive time consumption, inconsistency, and insufficient support for discovering new emojis and selecting appropriate ones.
• Significance:
  • Emojis are an essential part of daily online communication, adding emotional and contextual information to interactions. However, barriers faced by visually impaired users often limit their social experience and communication efficiency.
• Research Motivation and Related Work:
  • Voice input is a commonly used and satisfactory communication method for visually impaired users. Existing voice-based emoji input methods only support word-level input, failing to meet the needs of complex emoji expressions and exploration of new emojis.
  • Related studies primarily focus on emoji usability issues, with limited systematic efforts to improve emoji input methods.

Solution

• Method or Solution:
  • Developed an innovative voice-driven emoji input system—Voicemoji.
  • Voicemoji features the following capabilities:
    • Supports semantic-level emoji search driven by voice input.
    • Provides context-based emoji recommendations.
    • Allows voice modification of emoji color or skin tone.
    • Offers fuzzy search and multiple options to facilitate the exploration of new emojis.
• Innovations:
  • Supports natural language input without requiring precise memory of emoji names.
  • Provides context-aware emoji suggestions, helping users discover new emojis and enhance their expressive capabilities.
  • Designed for global use, supporting both Chinese and English languages to serve multilingual users.
• Implementation Steps and Key Technologies:
  • Utilized Google Cloud Speech Recognition API for voice transcription.
  • Leveraged Dango API and DeepMoji model for emoji recommendations.
  • Based emoji search and recommendation on Emojipedia.
  • Users can employ various voice commands, including direct voice insertion of emojis and changing emoji skin tones.

Research Outcomes

• Specific Results:
  • Voicemoji significantly reduced emoji input time, achieving a 91.2% reduction compared to the Apple iOS keyboard.
  • The provided emoji recommendations were deemed highly relevant to users' voice input, and users found the recommendation feature very helpful.
• Advantages:
  • Voicemoji is faster, easier to use, and more favored by users compared to the traditional Apple iOS keyboard.
  • Enhanced the online communication efficiency and experience of visually impaired users, enabling more confident and enriched expression.
• Experimental or Evaluation Results:
  • User study results showed that participants preferred using Voicemoji during tests and began to adopt it spontaneously in daily life.
  • Voicemoji's usability score (SUS) was significantly higher than that of the iOS keyboard, while its workload score (NASA TLX) was significantly lower.
• Limitations and Future Directions:
  • The current implementation relies on online services (e.g., Google API), meaning network latency may affect actual user experience.
  • Testing was limited to iOS users, with no evaluation of Android users' performance.
  • Long-term field studies to verify users' sustained usage in real-world scenarios have not yet been conducted.
  • Future directions include developing Voicemoji as a keyboard application integrated into mobile systems, providing detailed explanations for emoji usage, and extending its functionality to other visual media (e.g., stickers and memes).

Through Voicemoji, this paper not only provides an innovative solution to the emoji input challenges faced by visually impaired users but also offers insights for designing broader accessibility input technologies.