Frequent corrections signal systematic bias in sensors or models

What it is

User corrections are not just usage feedback — they are free bias telemetry: every "you got that wrong" is a labelled sample of "model diverges from reality". Scattered corrections are individual noise (genuinely atypical situations); frequent, clustered corrections are the signature of systematic bias — a sensor in the wrong position, training data that never covered this population, a threshold set too hair-trigger. Read as load, corrections are a burden; read as telemetry, a resource.

The dividing line is the clustering pattern: corrections scattered near-uniformly across hours, rooms, and users are mostly the intrinsic uncertainty of inference (handled instance by instance); corrections clustering on an hour range, a room, a population, or a sensor combination point to a localisable, fixable systemic cause.

Why it happens

Why does clustering point at system causes? Because the signature of systematic bias is condition dependence:

• Sensor-layer bias clusters by space and time — a temperature sensor beside the air-conditioner vent errs mainly during cooling; a motion sensor half-blocked by furniture errs mainly for that sofa's sitting direction. The shape of the correction cluster is a portrait of the sensor's physical situation.
• Model-layer bias clusters by population and situation — for users with night shifts, absent from training data, "home in the evening = winding down" fails systematically; corrections will concentrate on those users' hours. The cluster shape is a portrait of the training-data gap.
• By contrast, the corrections generated by intrinsic uncertainty are near-uniform: the random error of the inference chain does not pick hours, rooms, or people.

This distinction separates "handle per-instance" from "fix at the root": the uniform background is what the correction channel digests (its normal job); clustered signals escalate to sensor review and model retraining — without escalation, one user pays the correction cost for the same system fault again and again until they quit (precisely the trust-loss path).

Studying it

• Machine learning's error analysis and concept drift detection provide the method frame: slicing error rates along condition dimensions localises biased subgroups; drift detectors (statistical monitoring of error rate) separate "stable bias" from "drifting over time". The correction stream is a ready-made error signal source; the methods transfer unchanged.
• Fairness research's subgroup error analysis is exactly isomorphic: population-sliced false-positive/negative differences locate for whom the model fails — "fails night-shift users", "fails large households" are the same problem in ambient-computing form.
• Evaluation paradigm: inject a bias with known conditions (skew one room's sensor confidence) and test how quickly and at what sample size the clustering analysis localises the biased condition. Localisation sensitivity (samples needed) and precision (granularity of the identified slice) are the two key metrics.

One methodological caution: correction streams carry self-selection bias — users correct only when an error is "worth mentioning"; small errors go unreported. Clustering conclusions are directionally reliable (where corrections cluster, bias exists), but absolute magnitudes underestimate. After fixing one cluster source, the relative ranking of remaining clusters matters more than raw counts.

Where it stops holding

• Frequent corrections can also reflect inflated expectations. The judgement meets its designed accuracy (say, 90%) while the user expects 100% — correction rates run high with no fixable bias source. The remedy is expectation management (explaining the intrinsic uncertainty), not the sensor; separate the two causes by "clustered or not" before deciding which side to move.
• Corrections from co-residents can cancel out. Two people alternately correcting the same judgement in opposite directions (one says too sensitive, one too sluggish) reads as high frequency but is really missing per-person models — split by person before reading clusters; don't rush to move the global threshold.
• Low-consequence judgements yield sparse corrections. Users don't bother correcting harmless mistakes; their telemetry under-covers. "Few corrections" cannot be read as "no bias" — only as "bias unreported".

Applying it

• Store corrections structured: besides the judgement content and outcome, persist the condition dimensions at the time (hour, room, occupants, sensor combination, judgement confidence) — a correction stream without condition dimensions cannot be cluster-analysed; it is just a pile of complaints.
• Run a correction-clustering dashboard: heatmaps of correction frequency sliced by condition dimensions, with drift alarms (a slice exceeding baseline by several standard deviations). The heatmap is humans hunting patterns; the detector is machines standing watch; run both.
• Wire clustered signals to an escalation path: corrections on one condition slice beyond threshold auto-generate a ticket (review that sensor / collect and retrain for that subgroup) instead of lying in logs awaiting accidental discovery.
• How to check: injection drills — deliberately bias a sensor's readings and measure the full-chain latency from bias onset through cluster alarm to generated ticket, plus localisation accuracy; after the fix, that slice's correction rate should fall back to the global background — the drop is the measure of the repair's yield.

Related

• Same group: Z2.08.1 Correction entry points must appear the moment the judgement takes effect · Z2.08.2 Corrections must update future judgements, not just the current one · Z2.08.3 Uncorrectable judgements lose user trust
• Nearby: Z2.01 Limits of sensors · Z7.04 Long-term evolution
• Search terms: error analysis · concept drift · subgroup performance · corrective feedback