Explanations must name the concrete signals behind a judgement

What it is

Explaining a context judgement works only when it names the concrete signals it rested on: "the living-room motion sensor has read no movement since 9 p.m. and your phone is home, so you were judged asleep" is an explanation. "Based on your usage patterns" is not — that repackages the judgement without adding anything examinable.

The test of a real explanation is falsifiability: the user can hold it against reality ("but my phone was charging untouched and I've been in the study all evening") and locate the error. A vague explanation cannot be argued with, so it cannot move the conversation toward correction — it only manufactures an impression of intelligence, degrading explanation into rhetoric.

Why it happens

Why are concrete signals the active ingredient? Because the user's way of checking a judgement is comparing evidence, not appreciating reasons. A wrong judgement has two possible failure sites: the evidence itself (a misread sensor), or the inference (evidence right, conclusion leaps). Naming the signals lets the user separate them:

• Reading "no movement on the motion sensor" while sitting in the living room all along localises an evidence error; the corrective action is repositioning the sensor or adding a source.
• Reading "phone home + late hour + no movement → asleep" while quietly reading localises an inference leap (still and late is not asleep); the corrective action is changing the rule or adding a confirmation step.

A vague explanation hides the boundary between the two failures, so the user can localise neither — the only available conclusion is "it's wrong", and the next step is usually not correction but switching the feature off.

Concreteness has a second function: signals are the anchor for action. A correction entry point needs to know "what to change" — which sensor, which rule, which threshold. An explanation without signals leaves the correct button with nowhere to point.

Studying it

• Lim, Dey and Avrahami's CHI 2009 work compared explanation types directly: for why-triggered and why-not scenarios, input-based why / why-not explanations improved the accuracy of users' understanding of system behaviour more than abstract summaries. Why-not explanations lean on signals especially hard — "which condition went unmet" must be itemised.
• Bellotti and Edwards's accountability frame states the requirement more sternly: the system must not only display its judgement (intelligibility) but expose the sensing it rests on and submit to questioning (accountability) — a system that reports conclusions without grounds is unaccountable by construction.
• Experimental paradigm: pair the same judgement with three explanations (none / vague / signal-level) and measure prediction accuracy (can users forecast the system's next behaviour), correction accuracy (do they change the right thing), and trust scales. Correction accuracy is the most discriminating dependent variable — users given vague explanations frequently amend the wrong object.

One methodological caution: test the degree of concreteness under real fault pressure. With no faults, every explanation version looks credible; the differences appear only once judgements fail. Designs must include trials where the judgement is wrong, or the study measures nothing but copy preferences.

Where it stops holding

• Concreteness has an exposure cost. Signal-level explanations hand over the household's sensor inventory and placement — transparency to the owner, reconnaissance to a guest or an adversary. Tailor granularity to audience: this entry claims "concrete enough to be falsifiable"; which layer, shown to whom, is the separate trade-off handled next.
• Signals are not the whole basis. Learned-model judgements (activity recognition) have no itemisable rule list; they can only surface the top-contributing signals (feature attribution). Such "most relevant signals" explanations are approximations — labelling the approximation honestly beats faking a complete account.
• Concrete explanation cannot compensate for insufficient sensing: however clearly explained, a judgement built on a single sensor is capped by that sensor. Explanation makes the problem visible; it is no substitute for multi-source verification.

Applying it

• Fix the explanation template in three parts: conclusion → signal list → inference rule ("judged asleep ← no motion since 9 p.m. + phone stationary at home; rule: both hold in the evening ⇒ asleep"). Missing the rule segment leaves "change the rule" corrections unactionable.
• In why-not scenarios, itemise every unmet condition; "conditions not met" alone is not allowed.
• Mark attribution-style explanations as approximate: "main factors: late hour, no movement, devices stationary" — stated as a contribution ranking, not the full rule.
• How to check: hand users a batch of known-wrong judgements and score correction hit rate (did they amend the right sensor / rule / threshold). Hit rate is the hardest acceptance metric for concreteness; "explanation satisfaction" alone would wave rhetorical explanations through.

Related

• Same group: Z2.07.1 The system's current context judgements must be viewable by users · Z2.07.2 Invisible context judgements make behaviour changes seem causeless · Z2.07.4 Overly fine explanations expose more sensor detail than necessary
• Nearby: Z2.08 Correcting context judgements · Z5.03 Tracing cause and effect
• Search terms: why-not explanations · intelligibility · feature attribution · explainable AI