Recognition failure needs explicit feedback and a manual fallback
Aliases: manual placement fallback · tracking state · relocalizing UI
What it is
No plane has appeared, yet the interface still says “tap to place.” The hand lets go, the object hangs, or it flies at the next tracking jump. Tracking failure feedback is the demand to say “no placeable surface is recognized right now,” and to offer a path that does not depend on this detection — pick a point, confirm a hand-drawn patch, or cancel.
Silence is not neutral. If the system keeps posing as “you may place” while it is failing, people will file the aftermath as their own bad aim.
Why it happens
The runtime already has states: initializing, insufficient features, moving too fast, relocalizing, normal. ARKit’s limited reasons and ARCore’s failure reasons keep them apart; application layers often fold them into one “still loading.” Folded together, people cannot pick the matching action — wait, change surface, or go manual.
A manual fallback works because it swaps the attach target from “a plane the tracker found” to “a surface the user declared.” The declaration can be coarse (a tap on a wall, a tap on the floor). Precision is worse than a good detection, but the pose has an owner, and later tracking at least knows which patch to lock. Without that path, failure can only retry, and retry on the same surface loops.
Studying it
Hide or show tracking state, and provide or withhold a “tap to name the landing” fallback. Score placement completion, wrong-attach counts, and whether people can say, after a failure, what to do next. Split prompt copy by the platform’s reason enum; do not run a single universal sentence.
Independent variables: visibility of state, presence of a fallback, whether the prompt matches the actual reason. Dependent variables: task completion, time from failure to recovery, share of users who produce an executable next step.
The ecological trap: in the lab, a prompt is obeyed. In the field, a prompt that covers the surface being looked at is dismissed at once, which is the same as no feedback.
Where it stops holding
In a pre-scanned room that always yields geometry, failure feedback is rarely used — it should still exist as the branch for a suddenly covered camera. Tools that are always manual (the user taps a surface every time) do not depend on detection success; feedback then answers “can this point be used,” not “detection failed.” Logging failure without putting it in the interface helps a researcher and equals nothing for the wearer. Children and first-time users need shorter state names than experts; experts can read “relocalizing,” first-timers cannot.
Applying it
- Bind the place control to “no surface recognized”: the control must not commit, and the copy must name a matching reason (too dark, moving too fast, try a patterned face).
- Offer a manual landing: the user taps a wall or floor, the object attaches to an estimated plane at that point, labeled “you chose this; it may be off.”
- Do not retry silently. After a few attempts, promote the manual landing to the main path instead of spinning.
- How to check: cover the cameras or aim at a blank wall and try to place. If the UI still says “placed” and the object hangs, the failure was swallowed. Add feedback and a tap-to-choose, then test again: people should get the object onto a surface they picked, without asking what to do.