A pattern of repeated failure suggests a systemic problem, not a flaky network blip
Aliases: systemic failure · not a blip · clustered errors
What it is
A one-off failure is a dropped packet, a single 503; backoff and hand-off are enough. Repeated failure is the same action, the same class of error, over a stretch of time — this page hanging on every open, every upload a 413, every write in the app a 401. The pattern says: the net did not twitch. Configuration, quota, auth, client version, or the other side in maintenance did. Keep treating each as a blip and people idle on “try again”; the strategy that actually has to change (sign in, upgrade, pick another file, wait out maintenance, find a human) never appears.
Why it happens
Blips and systems are generated differently. Retrying a blip succeeds sometimes in the distribution; retrying a system has success near zero, and time will not save it. People cannot read a distribution from one failure; the product has to fold many into a pattern: a count, clustering on the same code, the same endpoint failing in a row. Once it reads as a pattern, copy and action must shift gears: from “try again” to “this may be an expired session / a file over the limit / the service in maintenance”, with that gear’s action. Without the shift, nearby retry is still correct, but correct at the wrong problem layer — a local hammer on a structural nail.
Patterns can also cross modules. Main column, rail and submit all 401 is a dead session, not three independent load failures. Retrying each splits one person into three logins. Rising to the session layer is the response to the pattern.
Where it stops holding
“Repeated” on a weak network may still be a dense blip; the pattern should be breakable by the network coming back: once a stable connection restores success, do not escalate to “the system is broken”. A clock-wrong client causing certificate errors looks systemic; what to fix is time, not retry. Errors that are systemic on the first shot (an explicit maintenance page, a forced upgrade) need not wait for a pattern to form. Do not use pattern to frighten someone who failed twice by chance; the threshold has to clear “dense blip”, for example n in a row on the same endpoint in the same session, or the same auth error across modules.
Applying it
- Record consecutive failures by session, endpoint and error class. Past a threshold, change copy and the primary action; do not remain on “try again” forever.
- The same auth or maintenance code across modules rises to a session- or page-level explanation; put away per-module retry.
- Offer the exit that matches the pattern: sign in again, pick a smaller file, open a status page, contact support. When the network returns, clear the pattern and go back to blip handling.
- How to check: fail the same upload with 413 five times. If the fifth still only offers “try again” and will send the same file, the pattern was not read. Volume/format copy and a file change should appear. Then make the whole app 401: one sign-in, not a retry on every module.
Related
- Same group: I2.13.1 Automatic retries need backoff; immediate tight retries pile onto the server · I2.13.2 After several automatic retries fail, stop and hand the decision to the person · I2.13.3 Be explicit whether retry reuses the original parameters or allows the request to be edited
- Nearby: I2.08 Load failure · I3.13 State-machine completeness and illegal states · I1.06 Timeout strategy
- Search terms:
systemic failure·error pattern·not a blip