Learning Cooperative Personalized Policies from Gaze Data

An ideal Mixed Reality (MR) system would only present virtual information (e.g., a label) when it is useful to the person. However, figuring out when a label is useful is challenging; it depends on a variety of factors, including the current task, previous knowledge, context, etc. In this paper, we propose a Reinforcement Learning (RL) method to learn when to show or hide an object’s label given eye movement data. We demonstrate the capabilities of this approach by showing that an intelligent agent can learn cooperative policies that better support users in a visual search task than design heuristics. Furthermore, we show the applicability of our approach in realistic environments and use cases (e.g., grocery shopping). By posing MR object labeling as an RL control problem we can learn policies implicitly by observing users’ behavior without requiring experience sampling or any other form of supervision.