Supporting Task Switching with Reinforcement Learning

Document Title

Supporting Task Switching with Reinforcement Learning

Document Information

• Subject Area: Applications of Human-Computer Interaction Technology, Task Switching, and Attention Management
• Keywords: Task Switching, Attention Management Systems, Reinforcement Learning, Human-Machine Collaboration, Multitasking, Cognitive Models, Artificial Intelligence, Experimental Research, User Interface Design, Workload

Research Background and Problem Statement

• Identified Problems or Challenges:

  • Human attention resources are limited, yet the pressure in modern multitasking environments is unprecedentedly high. This can lead to accidents, errors, and socioeconomic losses.
  • Multitasking and task switching incur "switching costs," including time and effort, while external interruptions may exacerbate stress, cause errors, and reduce overall performance.
  • Although theories and concepts, such as "timely interruptions," have highlighted the need for Attention Management Systems (AMS), existing AMS have not been widely developed or applied in real-world environments.
  • Traditional AMS face issues such as narrow scope, lack of flexibility, and complex architectures, making them difficult to adapt to dynamic task-switching environments.

• Significance:

  • Providing users with practical attention management tools can improve task-switching efficiency and task performance, reducing the negative impact of switching costs on cognitive processing.
  • The development of AMS is critical for fields requiring rapid decision-making and responses (e.g., driving, manufacturing).

• Research Motivation:

  • To explore AMS based on Reinforcement Learning (RL) and Cognitive Models (CR), aiming to identify optimal interruption timings and enhance user task performance.
  • This study seeks to address the gap between theory and practice by proposing a scalable solution to enhance human multitasking capabilities.

Solution

• Method or Approach:

  • A reinforcement learning-based attention management system is proposed, which observes the user's environment and automates task switching to optimize user performance.
  • The AMS is trained based on a user model, using Computational Rationality to simulate human cognitive constraints (e.g., visual limitations and reaction times).

• Innovations:

  • Combines reinforcement learning with cognitive models, training the system based on user behavior constraints to address the flexibility and complexity issues in conventional AMS.
  • Designed an innovative game task environment (a dual-platform balancing game) to simulate real-time multitasking scenarios for evaluating AMS effectiveness and user experience.

• Implementation Steps and Technical Details:

  • Training Phase: The AMS is trained using a user model with cognitive constraints, enabling it to develop optimal switching strategies based on environmental states.
  • Experimental Conditions: Four different modes were set, including automated task switching with AMS and user-driven switching strategies.
  • Model Refinement:
    • Unconstrained Model: Simulates superhuman performance, ignoring cognitive limitations, used as a baseline for comparison.
    • Cognitive Model: Designed with constraint parameters, such as visual perception errors and delays, to simulate user behavior.
  • Experimental Tools and Platforms: Developed using the Unity game engine and reinforcement learning frameworks (ml-agents library).

Research Findings

• Specific Results:

  • The AMS based on the cognitive model achieved significant performance improvements. In the multitasking game, the system enabled players to achieve significantly higher scores compared to their self-directed strategies.
  • Users reported reduced subjective workload, with improved scores in psychological, physical, and time demand metrics.

• Comparative Advantages Over Existing Solutions:

  • The automated switching strategy of AMS outperformed user self-directed decision-making, particularly in tasks requiring continuous rapid responses, significantly improving task performance.
  • Compared to the unconstrained model, the AMS trained with cognitive constraints better aligned with actual user performance, enhancing collaborative effectiveness.

• Experimental and Evaluation Results:

  • Players using the cognitive model-based AMS achieved average scores approximately 1.5 times higher than those using notification or self-switching modes.
  • Workload scores under different experimental conditions showed that the cognitive model-based AMS significantly reduced psychological and physical demands.

• Limitations and Future Directions:

  • Limitations: The experimental dual-platform task environment was relatively simplistic, and the AMS was trained using a single cognitive model, not accounting for diverse individual characteristics.
  • Future Directions:
    • Develop personalized cognitive models based on individual data to accommodate different users.
    • Further improve task diversity and cognitive models, such as incorporating physiological measurements to optimize workload and stress.
    • Explore AMS applications in other complex multitasking scenarios (e.g., drone management, airport control tower simulations).

Conclusion

This study demonstrates that an attention management system based on reinforcement learning and computational rationality can effectively improve human multitasking and task-switching performance. By integrating cognitive constraint models with RL, the system significantly enhances task scores while reducing user workload. Future research will focus on task diversification and personalized system design to achieve broader applications and optimizations.