Automation keeps adjusting its strategy from observed behaviour

What it is

The other half of mutual shaping: the system also adjusts its strategy from what it observes — learned scheduling (when to pre-heat), habit modelling (usual arrival time), personalised thresholds (how bright is too bright). This is the essential difference between ambient automation and fixed-rule automation (the old timer, the photocell switch): with the latter, users adapt one-way to an unchanging rule; with the former, two adaptors chase each other.

Because the "user adapts to system" and "system adapts to user" streams run simultaneously, such a system's behaviour cannot be read off the design document alone — the system running three months in and the system on installation day no longer share a strategy.

Why it happens

The system-side pipeline: behaviour observation → pattern extraction (clustering "typical days", frequent slots, preferred ranges from daily data) → strategy adjustment (lead times, thresholds, trigger conditions) → behavioural output. Each stage injects a modelled assumption about "the user"; assumptions steer adjustments, adjustments change outputs, and outputs feed the next round of observation.

Two simultaneous adaptors make a feedback loop: the system's adjusted behaviour changes the user's subsequent behaviour, which enters the system's next learning round. Three dynamical regimes:

• Convergent: increasingly smooth; the two meet at the user's true preference — the design ideal;
• Oscillating: the user tunes, the system learns, the system changes, the user retunes — nobody settles; typical when both sides are too reactive;
• Locked-out: after a user abandons a function, the system reads "not used" as "not wanted" and further down-weights it — the function is slowly starved by the learning mechanism, though nobody ever decided to switch it off.

There is also a built-in asymmetry: users see the system's every output (and can react), while the system's observation of the user is panoramic and silent — the two sides hold utterly different pictures of "which round of adaptation are we in". The user believes the system is still as installed; the system has re-learned three times.

Studying it

• Intelligent/adaptive interface research: Höök's 2000 argument represents the literature's consensus — adaptive interfaces carry a possible performance benefit alongside costs in unpredictability and loss of control, and whether the benefit materialises depends on the visibility and controllability of the adaptation. Two decades of empirical work have largely preserved that structure of conclusion.
• Method: adaptation-curve analysis on long deployments — place the system's strategy-adjustment points and the user's behaviour-change points on one timeline and classify the regime as converging / oscillating / locking out; the locked-out signature is a monotone decline in a function's usage running together with a decline in its system-side weight.

Where it stops holding

• The predictability cost of drift is not this entry's subject (it belongs with predictability); the focus here is the bidirectional structure. One sentence where they cross: adaptation frequency must fall within what users can digest — adjusting on the data's rhythm (daily) while being perceived on the event's rhythm (per surprise) yields exactly "why did it change again".
• Not every dimension should be learned. Identity, permissions, and safety-relevant behaviour must not be optimised from observation — the system should never "learn" to help a member slip past a curfew. The boundary of learning is a product decision, not a question of model capability.
• Small-sample overfitting is endemic. Household data is small and calendar events perturb it (holidays, guests, overtime); a single late arrival read as "new routine" is a household regular — pattern extraction needs minimum-evidence constraints.

Applying it

• Throttle system-side adaptation: adjust at a granularity users can narrate (weekly, not daily); a single event may enter "observation" at most, never directly change strategy.
• Make adaptation outputs state their evidence: "based on your usual 18:40 arrival over the last three weeks" — surface the assumption and give the user a chance to correct it on the spot; the correction is itself a clean training signal.
• Maintain an explicit no-learn list: safety, permissions, and behaviour affecting others are excluded from observational learning, written into the product's principles.
• How to check: two-column timeline review — beside every strategy adjustment, annotate the user-behaviour evidence that triggered it; adjustments without sufficient evidence count as overfit samples and must be zero.

Related

• Same group: Z3.07.1 Users adjust their behaviour to fit how automation judges them · Z3.07.3 Mutual adaptation can entrench a wrong initial assumption · Z3.07.4 Long co-evolution needs periodic review, not unchecked drift
• Nearby: Z3.05.4 Behavioural drift in learning automation keeps eroding predictability · Z3.04.3 Overrides should be learned by the system
• Search terms: adaptive automation · personalisation · feedback loop · adaptive user interfaces