Higher initiative makes system behavior harder to predict and raises the cost of error

What it is

The more a system acts for someone, the harder it is for that person to say, before it happens, “what is next.” Unpredictability itself enlarges error: there is no time to block, and the blocking posture is not in the hands. High initiative upgrades a classification error from “a wrong suggestion” to “the world already changed.”

Why it happens

Prediction needs a stable mapping: same perceptual cues, same system act. As initiative rises, the mapping’s output shifts from symbols to writes on the world, side cues thin (no dialog), and people can only model from after-the-fact results. Automation surprise is that model breaking. Error cost scales with write depth: a wrong hint can be ignored; a wrong execution needs undo, social repair, or physical recovery. Physiological implicitness hides the triggering cue, dropping predictability another notch—people do not even know which cue to watch to forecast.

Studying it

In a sequence of rising initiative, measure “next-act prediction”: pause the system and ask what it will do. Factors: rung, cue visibility. Outcomes: prediction accuracy, recovery time after a surprise, irreversibility of the error. Field diaries catch “I did not think it would” better than laboratory ratings. Measuring only efficiency writes unpredictability as cleverness.

Where it stops holding

For experts on a highly repeated industrial flow, high initiative can remain predictable because the mapping has been trained into a skill. For guests, occasional use, and shared devices, that same high initiative is surprise. Some error costs are absorbed by the environment (pre-lighting a lamp nobody cares about); low predictability then is not a disaster. In safety-critical domains, unpredictability is unacceptable on its own, whatever the accuracy.

Applying it

• Before raising the initiative rung, raise forecastability: fixed rules, visible cues, a short catalog of acts.
• High-initiative acts must be blockable and undoable, pulling cost back from “already happened” to “still changeable.”
• If the cue is physiological and invisible to the user, keep initiative below an equivalent function driven by location or a button.
• Verify by interrupting just before an act and asking unbriefed users what the system will do; a high miss rate means that rung should not default to execute.

Related

• Same group: C9.12.1 System initiative can be graded; hint-only versus direct execution are different degrees of autonomy · C9.12.3 Default initiative should be conservative; users can opt into more autonomous behavior · C9.12.4 The fitting initiative level for one function can differ by situation and should not be globally uniform
• Adjacent: C9.06 Cost of Sensor False Positives · C4.02 The Midas Touch Problem
• Search: automation surprise · predictability · initiative cost