Unexplained automation gets attributed to malfunction
Aliases: attribution to malfunction · breakdown attribution · trust loss from silence
What it is
When an automation does something the user did not expect and offers no reason, users do not think "it probably has its reasons" — they default to "it's broken" or "it's misbehaving". Unexplained surprising behaviour is indistinguishable from malfunction in the user's eyes. This is the most direct behavioural consequence of missing intelligibility: the harder a system works on the user's behalf, the more often it gets taken for broken.
It is not a transient "users don't understand new tech" effect. Attribution is an automatic response to surprise: a violated expectation triggers a search for causes, and when the search has no input, the cheapest available answer is "it's faulty". That default needs no evidence — only the system's silence.
Contrast the same surprise with one line of reason attached ("heating lowered: no presence detected in the home") — it gets filed under "system working correctly". Surprise itself does not create fault attribution; surprised silence does.
Why it happens
Three steps.
A violated expectation triggers attribution. People hold predictions about a system's behaviour; anything outside the prediction (did not do what it should, did what it should not) creates a demand for explanation. Because an automation's routine behaviour mostly stays outside attention, the behaviours that do get noticed are disproportionately the surprises — the noticed sample systematically over-represents faults.
Absent explanations, the cheapest attribution wins. Attribution follows the path of least effort: whatever explanation is available gets used. When the system is silent, "malfunction" and "randomness" are the only candidates. Worse, negativity bias: unexplained events are more readily encoded in memory as negative ones, accumulating into a "this system is unreliable" narrative.
Once formed, the fault frame is self-reinforcing. Observing with the prior "it keeps glitching", the user reads even neutral events as symptoms; the trust discount reduces use and reliance, and the system lapses into a white elephant. The most practical part of this entry is the asymmetry: loss is fast, recovery is slow — even after explanations are added later, rehabilitation takes many failure-free experiences.
Studying it
- Attribution interviews: in smart-home field studies, ask users to recall moments when the system "behaved strangely" and reconstruct what they thought was happening. A robust finding: "it's buggy" is the first attribution even when the cause later proves to be a rule a family member changed — users prefer fault over "the system had a reason I don't know".
- Trust-calibration experiments: stage a single surprising-but-reasonable action (the system does something sensible the user never asked for) and compare versions with and without an accompanying explanation on trust scales, reliance behaviour, and subsequent prediction accuracy. The why / why-not experiments in the intelligibility line use exactly this skeleton.
- Diary / experience sampling: sample users' in-the-moment appraisals of system behaviour across a long deployment, tracking the baseline frequency of fault attributions and its correlation with disengagement.
One methodological caution: a one-shot experiment captures attribution differences but not the asymmetric recovery — fast loss and slow recovery need weeks of deployment to observe, and a cross-sectional survey will misread the recovery period as "all fine now".
Where it stops holding
- Only surprises trigger attribution. Behaviour consistent with expectations needs no explanation; explaining everything is interruption. The goal is not "explain all actions" but "no surprise goes unexplained".
- A trust reserve can absorb one hit. After a good track record, users may rationalise a single silent surprise ("it probably detected something"); but that is spending the reserve, not a free pass — two or three in a row collapse it just the same.
- A wrong explanation is worse than none. If the given explanation later proves false (the system said A, the truth was B), users stop trusting the explanation channel itself, and even true explanations get discounted afterwards — explanations must come from real logs, not public-relations phrasing.
Applying it
- The system should know when it does something rare: rank actions by historical frequency and attach explanations to the rare ones automatically. Cheaper than explaining everything, safer than explaining nothing.
- The explanation arrives with the behaviour, not afterwards — attribution completes within seconds of discovery; a statement the next day cannot reshape a judgement already made.
- Write concrete conditions and values ("lights off: nobody detected in the living room after 23:40"), not tendencies ("to optimise energy use") — the latter reads as a disclaimer and deepens suspicion.
- How to check: deploy one reasonable-but-surprising behaviour in two arms, with and without an explanation, and measure trust and reliance over the following week; then withdraw the explanation for a week and watch the decay rate. Attribution rate and recovery time are the explanation feature's real report card.