Loki: Facilitating Remote Instruction of Physical Tasks Using Bi-Directional Mixed-Reality Telepresence

Remotely instructing and guiding users in physical tasks has offered the promise of training surgeons remotely, guiding complex repair tasks by an expert, or enabling novel learning workflows. While it has been the subject of many research projects, current approaches are often limited in the communication bandwidth (lacking context, spatial information) or interactivity (unidirectional, asynchronous) between expert and learner. To address some of these issues we explore the design space of bi-directional mixed-reality telepresence systems for teaching physical tasks, and present Loki, a novel system which explores the various dimensions of this space. Loki leverages video, audio and spatial capture along with mixed reality presentation methods to allow users to explore and annotate the local and remote environments, and record and review their own performance as well as their peer’s. In this paper, we explore the design space of mixed reality telepresence for physical tasks, contribute the system design of Loki which enables transitions between the elements of this design space, and validate its utility through a varied set of scenarios