Reducing the Cognitive Load of Playing a Digital Tabletop Game with a Multimodal Interface

Title of the Paper

Reducing the Cognitive Load of Playing a Digital Tabletop Game with a Multimodal Interface

Paper Information

• Field of Study: Human-Computer Interaction (HCI), multimodal interaction design using voice and touch, and its impact on cognitive load
• Keywords: multimodal interface, voice and touch, cognitive load, interactive surface, comparative study, user experience, intelligent game interface, experimental design, digital tabletop game

Research Background and Problem

• Identified Problems or Challenges:
  • High cognitive load increases user stress, error rates, and limits the likelihood of task success.
  • Multimodal interfaces (MMIs) are theoretically superior to unimodal interfaces (UMIs), but their advantages lack systematic validation in practice.
  • There is a lack of comparative studies directly assessing the cognitive performance of MMIs versus UMIs and their effectiveness in different application scenarios.
• Significance of the Problem:
  • Digital tabletop games integrate complex tasks and real-time player interactions, making them ideal for testing cognitive load and user experience.
  • Selection and system control tasks are critical operations in games and other unconstrained interaction domains, requiring reduced cognitive load to enhance user experience and interaction efficiency.
• Motivation and Related Work:
  • Researchers have proposed and demonstrated the potential advantages of MMIs in certain scenarios (e.g., Oviatt's research on voice and pen interaction), but these studies are often based on experimental models or non-functional interface implementations.
  • Introducing multimodal voice and touch interaction into digital tabletop games can further validate the applicability of this interaction mode.

Solution

• Proposed Method or Solution:
  • Develop and evaluate a multimodal interface (MMI) combining voice and touch, and compare it with a single-touch menu-based interface (UMI).
  • Use a dual-task paradigm to measure users' cognitive load and employ user experience surveys to assess intuitiveness and satisfaction.
• Innovations:
  • First comprehensive evaluation of a fully functional multimodal interface (without manual input assistance).
  • Propose a systematic design approach to evaluate cognitive load through task performance on an interactive tabletop surface.
• Implementation Steps/Key Techniques:
  1. Digital Tabletop Game Development: A test platform, "Mice Dance," was implemented, where users select virtual mice and complete tasks (e.g., collecting cheese, disabling traps).
  2. Interface Design:
     • MMIs support simultaneous or sequential voice and touch input.
     • UMIs use a circular menu with short-press and long-press operations for task selection.
  3. Experimental Design:
     • A dual-task paradigm was used, requiring users to perform a primary task (touch- or voice-based) and a secondary task (physically moving markers to a target location) simultaneously.
     • Task execution times for both interfaces were tightly controlled to ensure comparability.
  4. Quantifying Cognitive Load and User Experience: Cognitive load was measured using SEA, PSR, and RTLX questionnaires, while user experience and interaction intuitiveness were evaluated with AttrakDif and QUESI questionnaires.

Research Outcomes

• Specific Findings:
  • MMIs significantly reduced cognitive load (supported by both subjective and objective data) and were more intuitive and user-friendly.
  • The advantages of MMIs were particularly evident under high task frequency conditions, with a significant reduction in secondary task omission rates.
  • Most participants preferred using MMIs, reporting smoother operational experiences.
• Advantages Compared to Existing Solutions:
  • By combining voice and touch input, MMIs reduced users' reliance on visual resources, enabling quicker detection of competing tasks.
  • MMIs leveraged humans' inherent communication abilities more naturally than UMIs, enhancing task operation intuitiveness.
• Experimental or Evaluation Results:
  • Cognitive Load: SEA and RTLX questionnaires indicated that MMIs significantly reduced users' workload, especially under high task frequency conditions.
  • User Experience: In the AttrakDif and QUESI questionnaires, MMIs scored higher across multiple dimensions, including usability and attractiveness.
  • Qualitative Feedback: Most users agreed that MMIs were more intuitive and smoother; however, a few noted issues with voice recognition errors.
• Limitations and Future Directions:
  • The study only included simple primary tasks; future research could extend to more complex strategic task scenarios.
  • The impact of MMIs in multi-user environments was not tested; future work should consider the potential influence of social interactions on cognitive load.
  • Limitations in current voice recognition technology may affect performance in complex task scenarios, warranting further research into advanced voice recognition algorithms.
  • Broader validation of generalizability is needed, particularly in contexts involving virtual reality or augmented reality technologies.