A Simulation Model of Intermittently Controlled Point-and-Click Behaviour

Title of the Paper

A Simulation Model of Intermittently Controlled Point-and-Click Behaviour

Paper Information

• Domain: Human-Computer Interaction (HCI), User Behavior Modeling, Click Task Simulation
• Keywords: Point-and-click behavior, simulation model, user performance, Fitts' Law, deep reinforcement learning

Research Background and Problem Statement

• Identified Issues or Challenges: Point-and-click behavior is a fundamental interaction task in modern desktop environments. However, existing user performance models primarily predict aggregate metrics (e.g., completion time and error rate) and fail to simulate the dynamic interaction process. Furthermore, previous models inadequately account for key characteristics of user control, such as predictive intermittent control, and the cognitive processes involved in clicking actions.
• Significance: A precise simulation model for point-and-click behavior can aid in optimizing user interface design, reducing development costs, and providing deeper insights into user behavior.
• Motivation and Related Work:
  • Existing models for point-and-click behavior fail to balance the dynamic interaction process with human cognitive and motor characteristics.
  • Recent advancements in control theory and dynamic modeling provide a theoretical foundation for more in-depth simulation (e.g., BUMP model and ICP model).

Solution

• Method or Solution:
  • A simulation model is proposed that integrates human visual perception, intermittent motor control, click decision-making, upper limb kinematics, and the effects of input devices on point-and-click behavior.
  • User behavior strategies are optimized using a Markov Decision Process (MDP) based on deep reinforcement learning.
• Innovations:
  • The model combines several existing submodules, including the intermittent motor control model (BUMP) and intermittent click planning model (ICP), to reproduce realistic user movement trajectories and click result distributions.
  • Deep reinforcement learning is employed to optimize user behavior strategies, enabling dynamic and realistic simulation.
• Implementation Steps:
  • The model consists of five core modules: visual perception module, motor control module, click action module, mouse module, and upper limb module.
  • Users are simulated in a discrete-time environment, accounting for perceptual noise, motor noise, and mouse coordinate interference.
  • Deep Q-learning is used to train user strategies, optimizing click decisions and movement prediction perspectives across multiple scenarios.

Research Outcomes

• Specific Results:
  • The model accurately reproduces real users' completion time, click error rate, and cursor trajectories.
  • Ablation experiments, where certain submodules of the model are removed, demonstrate the significant impact of perceptual noise and mouse acceleration features on user performance.
  • Subjective user evaluations show that simulated cursor trajectories are indistinguishable from real user behavior.
• Advantages:
  • Compared to existing static models, this model can handle dynamic targets, account for users' cognitive and motor characteristics, and provide more realistic simulation results.
• Experimental or Evaluation Results:
  • Simulated completion time and failure rate closely match real user experimental data (mean completion time: simulation 0.84 seconds, real users 0.89 seconds; failure rate: simulation 31.3%, real users 37.7%).
  • Ablation experiments reveal that removing visual noise significantly reduces click failure rate (22.4%) but slightly increases completion time.
  • Deep reinforcement learning consistently converges to optimal behavior strategies.
• Limitations and Future Directions:
  • The model does not reflect behavioral differences among individual users and is currently designed for average user behavior.
  • Simulation of preparatory actions between continuous tasks remains limited, and handling complex target movements requires further exploration.
  • Future work could expand the visual module to address complex target trajectories and incorporate eye-tracking models to study the impact of visual encoding processes on performance.

This study provides a solid foundation for the development of future point-and-click behavior models while uncovering the underlying mechanisms of user behavior. It opens new avenues for designing more efficient interactive systems.