Degree of automation allocates responsibility, but users usually do not know which tier they are on
Aliases: opaque LOA · mode awareness of automation · duty in the dark
What it is
Level on the ruler decides whose name is on a failure. If operators cannot read whether this moment is advise, execute-after-approval, or already running, allocation happens in the dark: people nod with an advisory mindset, the system books duty at execution. Users rarely know their automation tier puts the first crack of traceability in awareness — not that the clause was unwritten, but that on the spot no one can point and say “we are on this rung.”
A product that covers three rungs with one name, “AI assistant,” designed the crack.
Why it happens
Parasuraman’s four stages can sit at different levels; UIs often use one status light. People guess the current level from the last experience: it asked me just now, so I think it is still asking; this step has already sent. Duty clauses follow actual authority; mindset follows the guess; when it breaks the two sides disagree. If an adjustable level exists but the current rung is invisible, people cannot dial, and cannot refuse “I never took duty at this rung.”
Level assigns accountability — that is the mapping on the ruler. This is the mapping going blind on the spot. Blind, and a complete after-the-fact log will not restore which rung that person thought they were on.
Studying it
Pack advisory and execution stretches into one product, unmarked. Mid-task, probe “who is doing this step, whose is a failure.” Dependent variables: accuracy of rung judgement, gap between duty percentage and actual authority, whether they fail to stop when they should. Independent variables: whether the current rung is shown, whether the rung changes per step, whether copy uses one brand name.
Wrong rung judgement with duty still claimed by the person is the most dangerous combination.
Where it stops holding
Fully manual, or fully autonomous with the machine’s duty declared outward, the rung is clear. Adaptive level changes without display manufacture blindness on a cycle. The duty clock at shift handoff is another moment; this is rung awareness in everyday running. Traceability that records input and basis records events; rung awareness is whether the person was aligned when the event happened.
Applying it
- Each step shows the current rung: advising / waiting for release / executing. When the rung changes with the step, the display must change; one master light is forbidden.
- Crossing from advise to execute must be an unmissable state change, not only an internal permission change.
- Check: mid-way, ask “whose is a failure on this step, can you stop it now.” If both answers disagree with actual authority, the rung is still dark. Mark the rung and ask again — if they still disagree, mark and authority are not yet aligned; it is not a type-size problem.
Related
- Same group: L4.15.2 Traceability requires recording input, decision basis, and result together; missing one makes replay impossible · L4.15.3 Logs must be kept until accountability might arise, not only as long as debugging needs them · L4.15.4 Handing final confirmation to a person does not mean responsibility has been fairly transferred; the person must have the conditions to refuse · L4.15.5 When systems are chained, the responsibility chain breaks at the interfaces; each segment's boundary must be drawn in advance
- Nearby: L4.01 Levels of Automation · L4.04 Takeover and Handoff Design · L1.09 Intervention Points in the Human Loop
- Search terms:
levels of automation·accountability·mode awareness
Cards in the same group
- L4.15.2Traceability requires recording input, decision basis, and result together; missing one makes replay impossible
- L4.15.3Logs must be kept until accountability might arise, not only as long as debugging needs them
- L4.15.4Handing final confirmation to a person does not mean responsibility has been fairly transferred; the person must have the conditions to refuse
- L4.15.5When systems are chained, the responsibility chain breaks at the interfaces; each segment's boundary must be drawn in advance