A final result alone cannot locate the problem

What it is

The task ends, the person is handed a wrong or strange result, and with no steps left they cannot tell whether the error was in understanding, in picking the wrong object, or in one tool failure. A final result cannot locate the fault is the other use of observability: process is not only so one can stop in the moment; it is so one can point at the step afterwards.

“It is written wrong” is an evaluation. “Step three wrote customer B’s address into customer A” is a location.

Why it happens

Diagnosis has to trace a difference in the result back to one beat’s input and act. A black box folds the whole chain into one output; the difference can only be re-run as a bundle. If re-runs do not reproduce, even “look again” is gone. People then change the prompt, change everything, and hit a possibly fine fault with a coarse intervention. This is not timely intervention: timely intervention happens before a crossing; location happens after it is already wrong, and asks which step. Without a step record, location degrades into a guess.

If a final report’s “I did these” is invented after the fact, location is pointed at steps that never existed — truth is a later problem; here the demand is that something besides the finale must leave steps, or there is not even material to invent from.

Studying it

Hold the same error, compare: final output only, output plus a post-hoc summary, timestamped real steps. Dependent variables: rate of pointing at the correct fault step, mis-pointed steps, number of re-runs, whether the fix is local or whole-task. Independent variables: whether steps include objects, whether they include tool returns, whether the summary matches the steps.

Pointing at the wrong step is worse than being slow to fix, because the next intervention will hit a healthy part.

Where it stops holding

A one-step task has no “which step” to point at; the finale is the step. Steps so dense they become an unreadable log fail location too — that is density. A process window that exists during execution and is thrown away at the end still fails location — the window has to remain until diagnosis ends. Timely intervention uses the window while it runs; location uses the window that was kept.

Applying it

• Keep a step record for every task until the person closes diagnosis: tool, object, result, time. The final output page must jump back to the failing step, not keep only a report.
• Allow replay from a step or trim from a step, so the only move is not re-running the bundle.
• Check: plant one object mismatch, and at the end give only the final output. If they cannot say which step, the finale is still carrying diagnosis alone. Open the steps and ask again — if they still point wrong, the record still lacks objects or returns.

Related

• Same group: L4.08.1 An agent's execution process must be observable · L4.08.2 Visibility of intermediate steps supports timely intervention
• Nearby: L4.12 Task Progress and Intermediate-state Visibility · L4.13 Agent Failure Reporting and Escalation · L1.11 Reproducibility of Non-deterministic Output
• Search terms: fault localization · process trace · observability