"I Didn't Catch That, But I'll Try My Best": Anticipatory Error Handling in a Voice Controlled Game

Title of the Paper

“I Didn’t Catch That, But I’ll Try My Best”: Anticipatory Error Handling in Voice-Controlled Games

Bibliographic Information

• Research Area: Human-Computer Interaction, Voice User Interfaces, Game Design
• Keywords: Voice User Interfaces, Game Design, Error Handling, Voice-Based Systems, Voice-Controlled Games, Voice Interaction

Research Background and Issues

• Challenges:

  • Advances in speech recognition and natural language processing technologies have made voice user interfaces (VUIs) an important and rapidly evolving field, but these systems remain prone to recognition errors.
  • The application of voice interaction in entertainment and gaming is increasing, but user experience issues caused by recognition errors warrant in-depth investigation.
  • Error handling is a critical aspect of voice interaction system design, often involving situations where user commands are either unrecognized or misunderstood.

• Significance:

  • Voice control can enhance game immersion and player experience, especially when recognition errors are effectively addressed.
  • Designing effective error-handling strategies can significantly improve user satisfaction and reduce frustration caused by speech recognition failures.

• Motivation and Related Work:

  • Studies show that seamless error handling in voice user interfaces is crucial for user experience and acceptance.
  • While there is extensive research on voice user interfaces and recognition errors, the application of voice interaction in game design remains underexplored.
  • Games are highly goal-oriented activities, and recognition errors can significantly impact players’ goal achievement and disrupt the “flow experience” in gaming.

Solution

• Method or Solution:

  • Proposed an anticipatory error-handling method that automatically executes locally optimized actions to fulfill goal completion and avoid obstacles when the game fails to recognize user commands, without notifying the user of the recognition failure.
  • Designed and implemented a voice-controlled arcade game, “Listen, Sparky!”

• Innovations:

  • Unlike traditional repetition-based error-handling methods, anticipatory error handling executes “locally optimized actions” based on the current game state, reducing interruptions to game flow caused by recognition errors.
  • This is the first exploration of applying anticipatory error handling in voice-controlled games to improve user experience, expanding the design boundaries of existing voice user interfaces.

• Implementation Steps and Key Technologies:

  1. Developed a voice-controlled game prototype using Unity 3D.
  2. Utilized Google Cloud Speech-To-Text service to process player voice inputs.
  3. Programmed game logic to execute actions based on optimization algorithms when voice commands are misrecognized.
  4. Set up experiments comparing two error-handling strategies (traditional repetition handling and anticipatory error handling).
  5. Conducted group user studies to evaluate user experience and system usability, including standardized questionnaires (SUS and PENS) and follow-up semi-structured interviews.

Research Findings

• Specific Results:

  • Usability scores of the game significantly improved under the anticipatory error-handling method (SUS score increased from 63.23 to 80.88).
  • Players perceived significantly fewer recognition errors, and their ratings of the intuitiveness of game controls also increased substantially.
  • While intuitive control showed significant improvement in user experience, there was no significant difference between the two groups in terms of autonomy and immersion scores.

• Advantages and Comparisons:

  • Compared to traditional repetition-based error handling, anticipatory error handling reduced the disruption of user experience caused by recognition failures, maintaining game fluidity.
  • Although this method technically optimized responses to recognition failures, it did not always align with users’ original intentions, potentially causing negative impacts on user experience for some players.

• Experimental or Evaluation Results:

  • Participants in the anticipatory error-handling group perceived significantly fewer errors, even though the actual recognition error rate was not significantly different between the two groups.
  • When the error-handling method conflicted with user intentions, users expressed dissatisfaction despite the technically optimized outcomes.

• Limitations and Future Directions:

  • Limitations include a small sample size (N=34) predominantly consisting of male players, and the speech recognition system was not trained for non-native English speakers.
  • Future work could extend to other application domains of speech recognition, such as navigation, education, and healthcare, and incorporate user behavior prediction models to further enhance user satisfaction in error handling.
  • Game design should consider personalized preferences for error-handling methods, allowing users to select mechanisms based on their needs.
  • Explore more complex voice interaction scenarios and expand the voice command library to enhance the flexibility and applicability of game controls.

Conclusion and Impact

This study demonstrates the potential improvements in usability and user experience brought by anticipatory error handling in voice-controlled games through experimental research. It also highlights the necessity of modeling user intentions, providing valuable insights for the design of error-handling strategies in voice user interfaces. Additionally, the study inspires designers to explore deeper into personalized user needs and contextual matching in the future, aiming to achieve smarter voice interaction systems and immersive application experiences.