Efficient Human-in-the-Loop Optimization via Priors Learned from User Models

Paper Title

Efficient Human-in-the-Loop Optimization via Priors Learned from User Models

Publication Info

• Topic area: Human-in-the-loop optimization for interface design using model-informed priors.
• Keywords: Human-in-the-loop optimization, Bayesian optimization, meta-learning, user models, synthetic users, reinforcement learning, interface adaptation, virtual reality, keyboard optimization, novelty detection.

Background and Problem

• Problem / challenge: Human-in-the-loop optimization (HILO) often requires numerous iterations due to the lack of prior task-specific information, making it time-consuming and impractical for real-time applications. Existing methods relying on real user data for prior knowledge are costly and lack scalability.
• Significance: Efficient HILO is critical for personalizing interfaces in real-time, especially for tasks demanding immediate and stable performance, such as input techniques in VR.
• Motivation and related work: Prior approaches have used transfer learning and meta-learning to accelerate optimization by leveraging past user data, but these methods depend on real human data, which is expensive and limits scalability. This paper seeks to eliminate the reliance on real user data by leveraging synthetic user data generated from predictive models.

Solution

• Proposed approach: Human-in-the-Loop Optimization with Model-Informed Priors (HOMI), a framework that pre-trains optimizers using synthetic user data generated from parameterized user models, enabling efficient real-time adaptation.
• Novelty:
  1. Introduction of HOMI, which repositions user models as training resources rather than optimization targets.
  2. Development of Neural Acquisition Function+ (NAF+), a Bayesian optimization method with a neural acquisition function trained via reinforcement learning on synthetic data.
  3. Integration of dynamic multi-objective adaptation and a novelty-aware fallback mechanism in NAF+.
• Procedure and key techniques:
  1. Model selection: Identify parameterized user models relevant to the task (e.g., Fitts’ Law, typing error models).
  2. Synthetic user generation: Generate diverse synthetic users by sampling model parameters.
  3. Meta-BO training: Train the optimizer offline using interactions with synthetic users to learn generalizable adaptation strategies.
  4. Deployment: Use the pre-trained optimizer in real-time with real users, leveraging prior knowledge and live feedback.

Results

• Concrete findings:
  • NAF+ outperformed baselines in early iterations, achieving faster convergence in both synthetic tests and a user study.
  • In synthetic tests, NAF+ demonstrated superior sample efficiency, dynamic objective weighting, and robustness to novel users.
  • In a user study on mid-air keyboard adaptation, NAF+ achieved statistically better performance than TAF and ConBO in early iterations.
• Advantage over baselines:
  • Faster convergence compared to Transfer Acquisition Function (TAF) and Continual Bayesian Optimization (ConBO).
  • Robust handling of out-of-distribution users through a novelty-aware fallback mechanism.
  • Better early-stage performance compared to ConBO, which relies on gradual learning across users.
• Experiments / evaluation:
  • Synthetic tests: Benchmarked NAF+ against baselines using a double-Sphere function and a soft keyboard typing simulation.
  • User study: Evaluated NAF+, TAF, and ConBO on mid-air keyboard adaptation with 12 participants over 10 iterations.
  • Metrics: Objective function combining typing speed and accuracy, running best performance, and NASA-TLX for subjective workload.
• Limitations and future work:
  • Dependence on reliable user models for synthetic user generation.
  • Limited exploration of other interactive systems and optimization strategies.
  • Future directions include integrating real user data for continual learning, extending to other applications, and leveraging advanced generative models for synthetic user simulation.

Summary

This paper introduces HOMI, a framework that pre-trains human-in-the-loop optimizers using synthetic user data to address the inefficiencies of traditional optimization methods. The proposed NAF+ method integrates reinforcement learning, dynamic multi-objective adaptation, and a novelty-aware fallback mechanism, enabling efficient and robust optimization. Synthetic tests and a user study on mid-air keyboard adaptation demonstrate that NAF+ achieves faster convergence and better early-stage performance compared to baselines. This approach redefines the role of user models in HCI, paving the way for scalable and adaptive interface optimization across diverse applications.