Q5.09.1staged rollout checkpointsdesignresearch

Rollout share usually increases by stage, each with a continue-or-pause gate

Aliases: staged percentage gates · progressive delivery gates

What it is

A canary is not a 1% cut followed by adding users by feel until 100%. The usual pattern is a staged increase in share: 1% → 5% → 20% → 50% → everyone, each stage with a written continue-or-pause checkpoint (gate). The gate reads whether observation inside that stage’s window crossed the continue threshold, not whether the calendar arrived. Unlike containing risk by keeping scope small, this is about how the percentage climbs and how each climb is gated. An increase without gates only delays a disaster and then enlarges it.

Why it happens

Climbing the percentage spreads uncertainty over time: an earlier stage exposes unknown failures on fewer people; a later stage takes a larger radius only afterward. Without a gate, climb is driven by release cadence or a market promise, and risk control exists in name. Gates need a window—too short and you see only novelty and deploy jitter; too long and harm accumulates while you wait. The observation set should also upgrade across stages: at 1% you watch crashes and severe errors; at 20% you first have enough volume for task completion or regional splits. Stages are not one smooth experiment; they are a sequence of conditional releases. If the last door did not open, the next door does not exist.

Studying it

Write the stage table as protocol: share, minimum window, metrics that must be seen, and a three-way call of continue / pause / roll back. After the fact, check whether the actual share curve obeyed the gates or was broken by “must be at 100% on Friday.” Report by stage; do not backfill post-full data into early stages as if they had already passed. Canary and progressive-delivery literature in software engineering stresses automated gating; HCI evaluation has to specify how experience metrics enter the same door, rather than leaving crash rate to engineering alone.

Where it stops holding

On a tiny user base, 1% may be a dozen people and the gate has no statistical meaning; use a sequence of named sites rather than a percentage myth. Regulatory windows and contract cutover dates can smash the stages; write exception clauses in advance. Some network-effect features never show real behavior at low share (social invites, inventory matching), so gates see “fine” until a critical mass. If only engineering on-call owns the gate and product and research are absent, experience failures never enter.

Applying it

  • Print the share sequence and each stage’s three-way call on the release ticket; dates are window floors, not automatic promotions.
  • Name product and research on-call for each stage; the gate must not be engineering-only.
  • After a pause, the retro must change the material or the monitoring; do not ram the same state into the same door again.
  • The leap to 100% uses the same gates as earlier leaps; arriving is not an exemption.

Related

  • Same group: Q5.09.2 Feature flags must toggle independently per user · Q5.09.3 Pilot users must know they are in a trial so they can interpret failures · Q5.09.4 Monitoring during rollout must cover exceptions and performance, not only business metrics
  • Adjacent: Q5.06 Canary release and pilots · Q3.04 A/B testing
  • Search terms: staged rollout checkpoints · progressive delivery · canary gate

Cards in the same group

Quick Actions

Share

Share this page

ios_share

https://hci.top/en/handbook/Q5.09.1