Prototype performance is not product performance
Aliases: prototype latency fallacy · lab speed is not ship speed
What it is
Interactive prototypes often run locally, on cached data, with no concurrency and no live network, so a click “returns at once.” Reading that smoothness as evidence the shipped product will be smooth is the prototype performance fallacy. Prototype performance can be better—no production queries, no permission layer—or worse: interpreted tools, huge design files, unoptimized motion. Either way it is not a measurement of post-release latency, throughput, or stability. Usability conclusions that rest on “feedback arrived in time to be read” can reverse on a slow network or a large data set.
Why it happens
Judgments of whether a system is usable depend heavily on the response window. Delay changes strategy: people re-click, abandon, switch entries, or blame themselves. If the prototype always sits inside that window, those strategies never trigger, so error recovery, cancel, and progress cues go untested. Conversely, stutter in a design tool can be misread as a product defect. Performance also couples to content scale: a list of ten demo records does not represent scrolling ten thousand live ones. Exporting lab completion times and satisfaction to production is measuring the same interface under the wrong physics.
Studying it
When a conclusion is about performance, treat latency, data volume, and concurrency as factors or at least as recorded conditions. Inject controllable delay, use near-production data volumes, and retest on the target network tier. Compare abandonment and compensatory behavior under a zero-delay build versus delay injected at production p95. Do not report completion time without hardware and network. If the only test is inside a design tool, scope the claim as “flow comprehension under immediate feedback,” not “the product is fast enough.”
Where it stops holding
Many early questions—are labels intelligible, can the entry be found—are insensitive to delay, and tests need not wait for “real performance.” Performance-sensitive cases are live collaboration, payments, search, media, and any control that uses millisecond feedback as confirmation. Weak mobile networks, geo-distributed backends, and cold starts are tails a lab prototype almost never simulates. Expert tolerance for delay also differs from novices; an internal “it’s fine” does not generalize.
Applying it
- Write the prototype’s runtime into the report: local or remote, record counts, whether delay was injected.
- For search, submit, and sync tasks, run at least one retest with production-like delay.
- Do not file “it felt smooth in the design tool” as a performance or stability finding.
- If delay cannot be injected, list performance as untested and schedule a near-production build.