Activity recognition is a probabilistic judgement

What it is

Inferring "what the user is doing" from sensor signals — activity recognition — yields a probability distribution, not a fact. Inside the system the judgement looks like "cooking 0.8, washing up 0.15, other 0.05"; on the product surface it is almost always rendered as a single certain label: "the user is cooking". That rendering discards the uncertainty, letting users and downstream automations consume a probabilistic judgement as settled fact.

The loss is not a display issue but a liability issue: the consequences of misinference — triggered automations, wrong personalisation — land on the design that treated the output as fact, not on the recognition engine.

Why it happens

Why it can only ever be probabilistic: sensors provide indirect evidence — acceleration patterns, current-draw signatures, sound spectra. One activity has countless ways of being performed (the signal difference between a quick stir-fry and a slow simmer may exceed that between cooking and laundry), while different activities share signal features (see the sibling entry on confusable activities). The mapping from signal to activity is inherently one-to-many; any output can only be a distribution.

The probabilities also drift after deployment: models are calibrated on data resembling the training set, while every household's individual differences — height, handedness, appliances, room layout — shift the effective confidence; temporal context further moves the judgement (the same kitchen sounds read as "cooking" at dinner time and "late-night snack" at 2 a.m.). Priors migrate with the situation, so the same signal decodes differently at different moments.

Studying it

• Benchmark studies: Bao and Intille (2004) used five body-worn accelerometers to recognise twenty everyday activities, reaching roughly 80% overall accuracy under controlled conditions — a reference point for the field's reliability scale: even at best, one to two errors in ten.
• Typical designs: independent variables include sensor count and placement, the granularity of activity classes (coarse classes are markedly more accurate), and the origin of training data (the participant's own vs. others'); dependent variables are accuracy and the confusion matrix. Cross-person generalisation is the pivotal variable: models trained on other people's data usually degrade substantially — which maps directly onto the smart home reality that per-person training is impossible.
• In-the-wild deployment: long-term wearing or environmental deployment in homes, comparing performance on controlled versus natural data.

Methodological caution: published accuracies come from evaluation sets drawn from the training distribution; product environments almost certainly depart from it. When using literature numbers for product decisions, assume the real error rate is higher than reported.

Where it stops holding

• Granularity determines reliability. Coarse judgements — moving/still, at home/away — can be made highly reliable; fine judgements — cooking vs. washing up — fall off a cliff. The finer the granularity a product depends on, the larger the bet.
• Confidence drifts. Moving house, new appliances, changed routines, a new household member — each invalidates calibrated confidence. Recognition quality is a continuing variable, not a one-time acceptance test.
• Confidence scores may be uncalibrated. A displayed 80% does not guarantee eight hits in ten; an uncalibrated score is worse than none, because it lends false certainty.

Applying it

• Tier downstream actions by confidence: execute automatically at high confidence, run reversible actions at medium, escalate to a question or conservative default at low.
• Phrase interface statements to avoid presenting inference as fact: "it looks like you've left" rather than "you've left" — leave room for denial.
• Provide a correction entry: let misjudgements be seen and corrected by users; correction records double as free data for evaluating recognition quality.
• How to check: sample inference logs against ground truth (household diary or camera review) to obtain per-class real accuracy and the most-confused pairs; make design decisions on measured numbers, not the engine's self-reported score.

Related

• Same group: Z2.02.2 Similar activities are hard to distinguish · Z2.02.3 High-stakes actions should not fire on a single inference
• Nearby: Z2.03 The costs of false positives and false negatives · Z1.03 Defining context
• Search terms: activity recognition · probabilistic inference · confidence calibration · smart home sensing