Unusual bodies and prostheses do not fit the skeleton the model carries in its head
Aliases: body prior mismatch · prosthesis tracking · non-normative pose · canonical skeleton
What it is
Inside a skeleton model sits a “standard person”: two arms, two legs, proportions and joint ranges from the majority body in the training data. Height at either end, a wheelchair, amputation, a prosthesis, scoliosis, a child's proportions, and that person will not fit. When it does not fit, the system does not say “I cannot see a body like yours.” It forces the fit, and emits wrong joints, a fake leg, or a crutch labelled as a forearm. Mismatch is a fight between the model's assumption and the body's fact, not a user who failed to stand canonically. Hand-shape variation hits finger counting. This hits the topology of the whole skeleton.
Why it happens
Most pose networks use a fixed topology (17 or 25 joints) plus a proportion prior. Missing a leg, the model still has to emit that leg's coordinates, so it grows one from a visible residual limb or from a chair. A prosthesis does not match skin statistics; depth and keypoints drop on sockets and metal, and joints land on the socket rim rather than the hinge. A wheelchair cuts visibility of hip and knee, and a standing prior invents a stance. Very tall or short bodies change segment ratios, so elbow and wrist land systematically off. Children have a different head-to-body ratio; an adult model puts the shoulders wrong. The wrong skeleton then enters gesture recognition, and full-body commands fail systematically for that group while mean accuracy still looks handsome.
Studying it
Stratify by body condition: wheelchair, unilateral amputation, prosthesis type, height quantile, children. Report joint error and gesture availability per stratum; do not dump them in “other.” If a stratum cannot safely perform full-body moves, report “unavailable,” not a low score. Control with optical markers or hand labels. Whether training data include the stratum should be a factor: how much error drops after adding those samples, and whether the majority stratum is hurt. Do not claim a universal full-body channel from walking young adults only.
Where it stops holding
A system that only uses near-field hands and never runs a full-body skeleton does not have this mismatch. Some prostheses carry their own sensing and the skeleton channel can be skipped. Switching from fixed to variable topology (joints may be absent) helps structurally, but the application must handle “there is no right elbow” rather than crash. Treating mismatch as an accessibility add-on, not as the channel's default test matrix, leaves the failure in the shadow of majority accuracy.
Applying it
- A full-body product's acceptance matrix must include wheelchair, missing limb or prosthesis, and children or height extremes. A missing stratum is unverified.
- Detect and declare “this body does not match model topology,” stop inventing joints, and fall over to hands or another modality rather than driving commands from a fake leg.
- Mean skeleton accuracy is not an accessibility pass. The pass is a usable path in every stratum, including announcing that this stratum will not use full-body.