Accuracy of pointing in a joint task depends on precise sync of participant positions

What it is

“That one” — a finger sends a ray. The ray starts at the speaker’s head and hand at that moment. If, in the other person’s scene, that head and hand are a few centimetres off or tens of milliseconds late, the ray lands on the neighbouring part. Deictic pointing alignment requires participant positions synced tightly enough that the same reference hits the same object in both views.

Whether the object is open or locked is state agreement. Whether the reference misses is whether people occupy the same coordinates at the same time.

Why it happens

A point is the intersection of a body-emitted ray with the scene. A wrong origin, a wrong direction, or an origin the other person sees from tens of milliseconds ago, and the intersection changes object. Near, densely packed targets amplify this: a centimetre of head error at fifty centimetres already jumps to the next screw. People also use pointing to build joint attention; one miss and the rest of the talk orbits the wrong object.

Sync has to be spatial and temporal. Colocation merges two origins into one shared frame; the pose stream still has to align sample times of head and hands, or the shared frame can be perfect and the point still rides a stale body. Drawing “their ray” from a locally guessed head pose is helping them point at the wrong thing.

Studying it

A pointing discrimination task: densely packed real or virtual targets, one person points, the other names which. Compare hit rate as position-sync error is increased (translation, delay, loss).

Error sources: head-position bias, hand-position bias, pose delay, target spacing. Scores: correct-object identification, neighbour-miss rate, spoken corrections, time from point to confirm.

Wide spacing (three posters on a wall) will barely show it. Pack targets until error jumps a cell.

Where it stops holding

One-to-one, objects uniquely namable by colour or number: pointing can be replaced by speech and the sync budget drops. Remote desktop-style collaboration has no shared physical ray; pointing becomes a cursor and this body-origin story does not hold. An avatar simplified to an orb has no usable hand origin; the pointing channel does not exist, and you should switch to explicit highlight rather than raise the sync rate. State inconsistency lets people point at the right object and still say the wrong open/close — that is an object-version problem; align the lid before talking about the ray. A weakest device with no hand cannot emit a ray it does not have; retarget pointing to a reticle or a touch pick.

Applying it

• Make pointing a ray or a landing highlight both sides can see, using the sender’s head and hand pose. Do not redraw it from a body the receiver is guessing.
• On dense targets, a point must briefly highlight the object that was hit. Do not leave the other person to intersect the ray themselves.
• When pose delay exceeds the pointing window, switch to click-to-lock: lock the object, then talk, rather than narrating along a jittering line.
• How to check: ten similar parts packed on a desk, one points, the other fetches. Add a noticeable head-position bias to the sync and fetches should become neighbour misses. After a landing highlight, misses should drop; if the highlight already sits on the right part and they still fetch wrong, that is naming or state, not pointing sync.

Related

• Same group：N5.10.1 Avatar pose is the main cue to others' intent; without it, collaboration drops · N5.10.2 Simultaneous operation of a shared object needs an explicit control-arbitration rule · N5.10.3 Network latency briefly desynchronizes how participants see the same object's state · N5.10.4 When devices of different capability share a room, the experience must align to the weakest
• Nearby：N5.05 Shared Space · N5.09 Spatial Anchors and Persistence
• Search terms：deictic pointing · joint attention · pose synchronization