Similar activities are hard to distinguish

What it is

Activity recognition errors are not spread evenly across activities but concentrate between signal-similar pairs: cooking vs. washing up, walking vs. sweeping, watching TV vs. reading while seated. Which pairs confuse is written in the confusion matrix — a standard evaluation artefact of any recognition engine, and it should equally be an input to automation design: if the two activities your automation must tell apart happen to be a confusable pair, misfires are not an unlucky tail event but a structural certainty.

"Similar" does not mean visually alike — it means alike in the sensor's signal space: two actions a camera easily separates can be one and the same curve to an accelerometer.

Why it happens

Three sources of similarity, each calling for a different remedy:

• Shared underlying features: cooking and washing up both happen at the sink side, both involve reciprocating hand movements, both carry water sound. Distinguishing information (the stove flame, the duration of running water) must actually exist in the signal before it can be exploited.
• Distinguishing information outside coverage: separating "watching TV" from "reading" needs gaze direction or the audio content of the room — physically present, but invisible to an accelerometer by principle. The information exists in the world but is not collected; a better model cannot help, only a different sensor can.
• The class boundary itself is vague: when does "cleaning" end and "tidying" begin is a matter of linguistic habit, not physical events — human annotators disagree with each other. This kind of confusion has no engineering solution, only a category-definition solution.

The corollary: some activity pairs no sensor can separate, because their physical signals are genuinely identical — through a wall, by sound alone, a person cannot tell wiping a table from kneading dough either.

Studying it

• Confusion matrix analysis: the standard first step of any recognition study. In Bao and Intille's (2004) twenty-activity results, confusion concentrated between activities with similar body mechanics (different forms of walking, household tasks with similar arm movements), confirming the structured nature of confusion.
• Real-home datasets: public smart home datasets (the CASAS series, among others) provide annotated daily activities under typical ambient-sensor deployments, letting designers check the separability of target activity pairs without building their own collection.
• Category definition studies: have multiple annotators independently segment the same activity recording and compare disagreement points — boundaries with high disagreement are category boundaries that are vague in principle.

Methodological caution: scripted laboratory collection artificially suppresses confusion (participants perform "canonical" versions with clean boundaries); transitions and parallel activities in natural behaviour (stirring while on the phone) are where confusion breeds — evaluate on natural data.

Where it stops holding

• Merging classes is a legitimate move. "Busy in the kitchen" is far more usable than "cooking": when two activities are inseparable in signal and the downstream action is the same, merging is both more honest and more reliable than forcing a distinction.
• The cost of confusion is situational. Some confusions are consequence-free (a midnight kitchen visit — water or snack, the light should come on either way); the ones that matter are those whose downstream actions differ. Price the account before investing in separation.
• Changing sensors solves only one case. It addresses "information exists but is not collected"; it cannot address "information does not exist" or "the boundary is vague".

Applying it

• Request the confusion matrix from the recognition provider, or measure one; cross-reference your automation inventory and mark which dependent activity pairs fall in which confusion tier.
• Redesign automations that depend on in-principle inseparable distinctions: merge the trigger conditions, or choose downstream actions that are safe under both interpretations.
• De-consequence confusable pairs: keep actions reversible, time-limited, or behind a lightweight confirmation.
• How to check: for each confusable pair on the list, replay real recordings (e.g. run the "cooking-triggered" automation during labelled "washing up" sessions) and count misfires — that figure is the automation's true error baseline.

Related

• Same group: Z2.02.1 Activity recognition is a probabilistic judgement · Z2.02.3 High-stakes actions should not fire on a single inference
• Nearby: Z2.01 The capability boundaries of sensors · Z5.06 Time windows and debouncing
• Search terms: confusion matrix · activity recognition · CASAS dataset · inter-annotator agreement