SpeakEasy: Enhancing Text-to-Speech Interactions for Expressive Content Creation

Research Background and Problem

• Issues or Challenges Identified by the Authors:
  Novice content creators face challenges when recording voice content for videos, including excessive time investment and numerous trials to produce high-quality audio. While existing Text-To-Speech (TTS) technologies can generate human-like voices, their interfaces are often overly complex or granular, making it difficult for users to create expressive voice content for videos.

• Why This Problem Is Important:
  Expressive voice content is crucial for social media engagement. High-quality audio helps content creators connect with their audience and enhances the appeal of their content. However, the inefficiency of current recording processes hinders creators' ability to publish content quickly. Furthermore, advancements in TTS technology have the potential to support users with language impairments or privacy concerns, making voice production more accessible.

• Research Motivation and Related Work:
  The authors explored the limitations of current TTS technologies, including mismatched emotional tone in generated voices, inefficient control interfaces, and users' difficulty in expressing personalized creative intentions (Ori study). Previous related work has made progress in analyzing emotional voice generation and increasing user control, but solutions that support high-level user feedback and diverse voice expressions remain lacking.

Solution

• Proposed Solution and Methodology:
  The authors designed SpeakEasy, an interactive TTS system based on the Wizard-of-Oz paradigm. SpeakEasy allows users to input additional contextual information embedded into TTS generation and provides high-level interactive features to support flexible voice iteration.

• Innovative Aspects of the Solution:

  1. SpeakEasy generates targeted voice content based on initial scripts and user-provided contextual information, effectively reducing the need for repeated trials.
  2. It offers convenient sentence-level iteration features, enabling users to adjust generated voices using simple adjectives or higher-level feedback.
  3. The system provides suggestions for various types of voice expressions, including "unexpected attempts," helping users expand their creative horizons.
  4. It supports voice selection and optimization through a simplified comparison interface.

• Implementation Steps and Key Technologies:

  1. User Input of Script and Context: Upload scripts in a text editor and add emotional or character-based contextual descriptions.
  2. Initial Voice Generation: Generate voice content based on the script and user-provided context to meet preliminary expectations.
  3. Sentence-Level Iteration and Modification: Iterate on individual sentences using recommended adjectives, diversity toggles, or free-text inputs.
  4. Comparison and Selection: Display iteration results in tabs, allowing users to easily compare and select the most satisfactory voice content.

Research Outcomes

• Specific Results:

  1. User satisfaction with SpeakEasy-generated voices was significantly higher than with two industry-standard tools (ElevenLabs and Speechify).
  2. Users could quickly and accurately express their intentions and achieve efficient voice iterations, while discovering new pathways to unlock creative potential.
  3. SpeakEasy not only excelled in ease of use but also significantly reduced cognitive load and time required for generating personally satisfying voice content.

• Advantages Over Existing Solutions:
  Compared to ElevenLabs' global sliding parameters, SpeakEasy's interface is more intuitive; compared to Speechify's fine-grained controls, SpeakEasy reduces the complexity of user decision-making. Additionally, SpeakEasy introduces "unexpected attempts" and sentence-level targeted interaction features, inspiring creativity and significantly expanding the possibilities of voice generation.

• Experimental and Evaluation Results:

  1. Users rated SpeakEasy as the best tool for initial voice generation and creative exploration.
  2. SpeakEasy effectively reduced task load (including cognitive burden and time consumption) while improving the naturalness and goal alignment of voice output.
  3. SpeakEasy excelled in helping users comprehensively compare generated voice options.

• Limitations and Future Directions:

  1. Challenges in Human Voice Usage: Current reliance on human-recorded voices may offer superior quality and acceptance, but this raises higher expectations for future real-time TTS implementations.
  2. Richness and Professional Control: While current recommendations sufficiently support general content creation, future iterations should include more professional editing features to cater to advanced user needs.
  3. Enhanced Prompt Input Exploration: Some users face difficulties when inputting initial context. Future tools could integrate recommendation and dynamic interaction mechanisms to help users create clearer prompt frameworks.

Through the implementation and validation of the SpeakEasy system, this research provides important guidance for designing emotional TTS systems and lays the foundation for future development in generative AI voice applications.