L4.08.1agent execution must be observabledesignresearch

An agent's execution process must be observable

Aliases: glass-box agent · watch it run · not a black-box finish

What it is

While an agent runs, a person must be able to see what it is doing: which tool, which object, which stretch of the plan. Agent execution must be observable requires a window onto the process, not only a result at the end. Execution that cannot be seen cannot be supervised; the level has in fact risen to full autonomy.

A spinner and “thinking” are not observation. Observation answers “what is it touching now.”

Why it happens

Supervisory control presupposes that the supervisor can sample the process under watch. If the process is sealed in a black box, the only sample points are start and end, and outbound acts in the middle have already happened without the person knowing. Endsley’s perception layer has no input during execution; out of the loop is not the person leaving, it is the window not existing. Observability is a yes/no: the window is open or shut. How to use it once open — density, truth, silent phases — is finer intermediate-state design; without a window those questions do not arise.

Products impersonate process visibility with a final summary because summaries are cheap and process streams are not. A summary answers “what got done,” not “what is it doing this moment.”

Studying it

Hold the same long task, compare black box (final result only), a state badge (“running”), and an object-level process stream (tool, object, state). Dependent variables: whether people can name the current object mid-way, whether they stop when they should, felt control. Independent variables: whether a window exists, whether refresh is timely, whether a log is dumped only at the end.

A mid-way probe “what is it doing” that cannot be answered is unobservable, however complete the log after finish.

Where it stops holding

A single short call, with the person watching the result appear, gains little from a process window. When tool detail must not be exposed to this operator, observability degrades to “which class of object, what kind of act,” not to saying nothing. That intermediate steps support timely intervention, and that a finale cannot locate a fault, are a use and a counterexample after observability holds; this entry only requires that the window itself exist.

Applying it

  • A long task must have a process view: at least current tool, current object, running / waiting / failed. A spinner cannot stand in for those three.
  • The process view is available during execution, not generated only after. Someone who left and came back should see this moment, not an empty state.
  • Check: halfway through, call the operator, cover the brand copy, ask “what is it touching.” If they cannot name an object, the window does not exist. Pasting a full log after finish does not repair that question.

Related

  • Same group: L4.08.2 Visibility of intermediate steps supports timely intervention · L4.08.3 A final result alone cannot locate the problem
  • Nearby: L4.12 Task Progress and Intermediate-state Visibility · L4.05 Interruptibility and Rollback · L4.14 Plan Visibility and Revision for Multi-step Tasks
  • Search terms: observability · supervisory control · agent transparency

Cards in the same group

Quick Actions

Share

Share this page

ios_share

https://hci.top/en/handbook/L4.08.1