Explaining the process and explaining the outcome are different things

What it is

A radiology pane that says “the model scanned high-density regions along the lung field,” and one that says “this scan is marked suspicious,” both sound like explanations. The first reports how the step was taken. The second reports which conclusion the step landed on. Process explanation and outcome explanation are not the same kind of talk: one answers what it did; the other answers what it takes the situation to be.

This entry only splits those two. Whether an after-the-fact story is the true mechanism, and whether one should explain this decision or the whole model, are different cuts.

Why it happens

When people ask why, they often stack two needs in one sentence. Miller’s survey of explainable AI notes that everyday explanation is usually contrastive about an event: why this outcome rather than another. That is the outcome layer. The process layer answers the path: which regions were looked at, which rules fired, where it stopped. The two layers serve different next moves. Knowing the path, a person can judge whether the run went off course and whether to change the input and rerun. Knowing what the conclusion means, a person can accept, rewrite, or hand off.

If the interface supplies only one layer, the gap is filled in as the other. A heatmap alone is read as a diagnosis; a “denied” alone is read as if the review process had been shown. The mismatch is of layer, not of word count.

Studying it

Pair the same outputs with two kinds of account: a process account (steps, regions, rule triggers) and an outcome account (label, score band, suggested action). Then give two kinds of task — predict the next similar output, or decide whether to accept this one. Independent variables: account type, task type (path prediction / outcome disposition). Dependent variables: prediction accuracy, disposition consistency, a subjective “I was explained to” rating.

Miller’s contrastive frame is used here as a task taxonomy, not as a finding that explanation always helps. Both accounts can raise the rating; subsequent action is what pulls them apart.

Where it stops holding

For an operator who only has to follow an instruction, the outcome layer is often enough and the process layer is interference. For an applicant who must change an input and resubmit, there is nothing to do without process. A process write-up that is only a log still behaves as an outcome — it supports no mid-stream intervention. This entry does not treat local versus global scope, nor counterfactuals of the form “if this field changed.”

Applying it

• Label which kind of question a pane answers. Process uses steps or “what was looked at”; outcome uses a conclusion and an action. Do not let one paragraph impersonate both.
• When the user must change an input, the process layer has to land on a changeable part (which document is missing), not on internal layer names.
• When the user only has to sign off, give the outcome layer and the consequence, not the training pipeline.
• Check: cover the copy and ask “what can you do now” and “do you know what it just did.” The two answers should match the layer you actually supplied; if people use process to answer an outcome question, the layers have mixed.

Related

• Same group: L5.01.2 A post-hoc explanation is not the true mechanism · L5.01.3 The object of explanation is the decision, not the model
• Nearby: L5.02 Local and Global Explanations · L5.08 Counterfactual Explanations · L5.05 The Moderation Principle of Transparency
• Search terms: process vs outcome explanation · contrastive explanation · why-question