A streak of successes pushes trust above the system's actual reliability

What it is

A coding assistant writes twelve functions right in a row; the thirteenth folds a drop-table into a migration. Those twelve did not change its base rate of “about one in five scripts will make trouble.” Handover, though, is already walking as if it can write scripts. A streak of successes pushes trust above actual reliability.

The overshoot comes from a run of ordinary successes, not from one destructive incident. The incident is a different class of event.

Why it happens

People estimate frequency from a recent sample. A streak washes failures out of the window; the estimate follows the window, not the long-run base rate. The interface further cheers the streak: every hit is shown whole, misses get a warning, so successes in the window are encoded more densely. On Lee and See’s calibration picture, the estimate drifts up with recent experience; if Muir’s update weights the recent more, the drift is harder.

The behaviour of overshoot is widening the scope of handover: from completing a line to editing a migration, from a draft to going straight to production. Scope widens while the base rate is unchanged, so the next miss often lands on the layer just widened — the system did not suddenly get worse; people lifted the task past the level it had always sat at.

Studying it

Hold base rate fixed, manipulate the length of the visible success streak, then have people choose the scope of handover on the next item (complete / edit the script / go live). Independent variables: streak length, whether a small miss is inserted now and then, whether a long-run base rate is shown. Dependent variables: gap between estimate and base rate, scope of handover, surprise at miss n+1.

The base rate must stay fixed. If the streak comes from a truly stabler version, what you measured is not overshoot.

Where it stops holding

On a task whose base rate is already extremely high and whose variance is tiny, a streak is evidence, not overshoot. When users can see a long-run count (right and wrong in the last hundred), the window is pulled open and this entry weakens. One severe injury emptying trust is the opposite direction, not overshoot. “No globally correct trust” is about the tick across tasks; this entry is about being lifted by a streak on the same task.

Applying it

• After a streak, before widening permission, show that task’s long-run base rate, not “twelve in a row.”
• Separate high-stakes acts (going live, changing a database) from low-stakes streaks; do not use the streak as a ticket to upgrade.
• Now and then insert a typical miss for this task into the success stream (a demo or a history), to break the window.
• Check: are streaking users handing the next item to a layer your base rate cannot hold. If yes, overshoot has happened; see whether their estimated success rate sits clearly above the long-run number.

Related

• Same group: L5.09.1 A reasonable trust level varies with task and situation; there is no globally correct trust · L5.09.3 Undertrust shows up as repeated manual checking, whose cost is often ignored · L5.09.4 Showing typical failures can suppress overtrust, at the cost of short-term adoption · L5.09.5 Trust is formed from personal use, not from statements and documentation
• Nearby: L5.03 Trust Calibration · L5.10 Asymmetric Effect of First Failures on Trust · L4.03 Automation Complacency
• Search terms: success-streak overtrust · recency in trust · overtrust