Attribution depends on identity features, not on the motion itself

What it is

The shape of a wave or pinch says which action occurred, not who produced it. Attribution has to consult identity features: relative stance, temporal continuity of a skeleton track, face and body pose, clothing color, a held beacon, or an explicit “I am operating.” Assigning “the motion that looks like select” to whoever currently has focus is treating gesture class as if it were a sender ID.

Why it happens

A gesture classifier reports distance from an anonymous template. When two people perform nearly identical rightward swipes, the class label cannot split them. Identity arrives on another channel: the tracker issues a temporary ID per skeleton, then binds that ID to a session user via position, appearance embeddings, or active registration. Temporary IDs swap after bodies cross—a routine vision-tracking failure—so “same skeleton throughout” is not enough; the system also needs features that can re-associate after a crossing. Without a stable identity, the system at best knows “a hand in the volume swiped,” not which account or undo stack to charge.

Studying it

Train two participants on the same vocabulary until the motions look alike, then have them perform simultaneously or interleaved in side-by-side, front-and-back, and crossing formations. Compare four attribution strategies: class-only, nearest-position, position plus appearance, and those plus an explicit claim. Dependent measures are identity accuracy and frames-to-recovery after a crossing. Score motion similarity separately so “the two motions were never alike” is not credited to the identity channel.

Where it stops holding

If furniture or floor markings constrain the volume to one standing place, position itself approximates identity and a fine face or appearance model adds little. Infrared skeletons can survive darkness where RGB appearance dies; masks, helmets, and uniforms weaken face and clothing cues. Active beacons (wristband, badge) can cover appearance failure, but excluding anyone without a beacon changes who is allowed to be a user. Look-alike household members and seasonal clothing shifts cut the lab accuracy of appearance embeddings.

Applying it

• Keep identity as a module separate from the gesture classifier: the classifier answers “what action,” the identity module answers “which person.”
• Provide a re-binding cue after crossing, turning, or a brief exit (relative position plus a short confirm). Do not assume a skeleton index is stable from entry to exit.
• Validate on paired samples where the motions are nearly identical and identity must come from non-motion features, rather than reading only the gesture-class confusion matrix.

Related

• Same group: C4.17.1 When several people share the sensing volume, the system must decide who owns the current action · C4.17.3 Misattribution treats one person's motion as another person's command · C4.17.4 Arbitration rules must be predictable to users, such as first-come or nearest-person
• Adjacent: C4.13 Full-body pose and skeleton tracking · C4.30 Bystanders and third parties
• Search: identity features · skeleton ID · person re-identification