Overrides should be learned by the system

What it is

Every override of an automation is an expression of preference — the system should treat it as a training signal, not discard it as noise or let it expire with the moment. The division of labour within this group: one entry settles "the user is obeyed this once", another "explicit wins on conflict"; this one handles "heard, and remembered". It converts explicit correction into implicit improvement: users vote with their actions, the system keeps the books.

The ledger has two possible objects, and confusing them produces wrong learning: parameters (comfort temperature from 24° down to 22°) versus rules (stop automating this time slot at all). A single override may be an exception (guests today) rather than a new preference (from now on) — and no system can tell which from one event. That is the central design problem.

Why it happens

The information structure of an override makes it high-quality training data: it carries context (when, under what conditions, who), a direction (what the user wants, not merely what they dislike), and proof of cost — the user paid an interaction cost to make this correction, and a signal someone pays for is an order of magnitude stronger than a thumbs-up.

But both ends of the learning dial are pits:

• Overfitting: permanently change default behaviour after one override. Guests came, so a normally-dark lamp got switched on — the system turns it on every day from now on, and the user is enraged by its overreaction.
• Underfitting: never learn. The user manually adjusts the same thing every Monday and the system ignores the regularity, and the user is enraged by its deafness.

Telling "exception" from "change of mind" requires accumulating evidence: only multiple independent signals harden into a new default; a single override stays on probation or triggers an on-the-spot question. Users' own language usually already contains the answer — "just this once" and "from now on" are different intents, and surfacing that distinction at the entry point is more reliable than making the system guess.

Studying it

• Field research on learning thermostats: Yang and Newman's 2013 in-home study of the Nest thermostat documented how users really interacted with "learning" — misunderstanding how it learned (believing it recorded audio, treating the leaf indicator as a reward to maintain) and inventing ways to "teach" it. The findings support two design requirements directly: the learning process must be visible, and what has been learned must be inspectable.
• Interactive machine learning: the general framework for incrementally updating models from small amounts of human feedback; its core consensus is that sample efficiency on small feedback sets determines usability.
• Method: paired deployments with and without learning, measuring the override-rate decay curve (learning works when overrides decline over time) and the mislearning rate (how often users protest "I never asked it to do this").

Where it stops holding

• Not every override is a preference statement. Some are reactions to failure (the automation itself misfired); some are temporary exceptions (guests, holidays). Learning a fault-driven override as a preference bakes the defect into the model — rule out "the system erred" as an explanation before learning anything.
• Signal ownership is ambiguous in multi-user homes. Whose override it was, and whose preference it represents — fail to separate them and one person's preference gets learned for the whole household (the night-shift member's late hours becoming the house default).
• Override learning needs disclosure. Users do not necessarily expect "my turning it off" to become training data; in privacy-sensitive domains (health, presence) silent learning is itself the privacy problem.

Applying it

• Offer a "what the system has learned from me" view: each learned adjustment listed, each one deletable — invisible learning is learning about someone in the dark.
• Learn conservatively: change default behaviour only after several like overrides or several days of them; a single override affects the moment only, or triggers an on-the-spot split — "just this once / from now on".
• Store exceptions and preferences on separate tracks: operations explicitly marked "just this once" go to an exception table with an expiry, never into the long-term model.
• How to check: track the override-rate decay curve and mislearning complaints; run a "single exception" test — after a user explicitly picks "just this once", no permanent change to system behaviour may follow.

Related

• Same group: Z3.04.1 Implicit paths cannot replace explicit control · Z3.04.2 Explicit actions should override automatic behaviour
• Nearby: Z3.06.4 Takeovers should be recorded to improve future judgements · Z3.05.4 Behavioural drift in learning automation keeps eroding predictability
• Search terms: learning from user feedback · preference learning · interactive machine learning · Nest thermostat