Portal-ble: Intuitive Free-hand Manipulation in Unbounded Smartphone-based Augmented Reality

How should we implement direct hand manipulation for smartphone-based augmented reality (AR) in larger spaces? We conduct an experience prototyping exercise to show that this may be harder than it might seem, as existing principles from prior literature were found to have issues with users. We identify issues in perception (moving to the object, reaching for the object), manipulation (successfully grabbing and orienting the object), and behavioral understanding (knowing how to use the smartphone as a viewport). Then, we introduce a combination of object-based feedback and accommodations to aid user interaction with virtual objects. Our resulting system implementation, Portal-ble, was more successful than baseline approaches in a user study for grabbing static and moving objects and for object manipulation, and was less mentally demanding. Put together, the Portal-ble system improves intuition and aids interacting with the virtual world in larger physical spaces.