Streaming shortens time-to-first-token, not total duration; it changes perception, not fact

What it is

Words arrive one by one and people feel “it has started.” The wall clock has not sped up: the last token’s time is almost the non-streamed completion time, sometimes later (a long connection to keep). Streaming changes perception, not duration. Selling it as faster passes time-to-first-token off as total time.

After locating which band the delay sits in, this says which needle streaming moves inside the band.

Why it happens

Waiting splits into “nothing yet” and “something already.” People mainly punish the first. Streaming compresses the punishment window to TTFT; punishment of total time is eaten by reading — waiting is also reading. That is attention accounting, not acceleration. Engineering for a stream may add buffers, retries, and batching limits, and duration occasionally gets worse.

If marketing and progress copy say “replies in seconds,” people will check the wall clock against total time. When it does not match, the perceptual gain is cancelled by the missed promise of faster.

Studying it

The same completion time, streamed versus given all at once. Measure: felt wait, satisfaction, actual completion time, reports of “faster.” Independent variables: TTFT, whether rate jitters, whether “all done” is marked at the end. Dependent variables: gap between felt and fact, early-leave rate.

Report two clocks: first token and completion. Felt wait alone will let streaming always win.

Where it stops holding

When people want to read while they wait (a long draft), the perceptual gain is the real goal; no apology is owed for unchanged duration, but copy still must not say faster. When the task is keyed to completion time (download the whole file, run a check), streaming did not help the task, only the mood. On a weak net, even TTFT may not be short. This entry does not discuss judging a prefix.

Applying it

• Split TTFT and total time as internal metrics. Outward, say “you see words first,” not “it finishes faster.”
• The end must have a completed state, so people do not read “still emitting” as “still a long time” or the reverse.
• Do not stretch total time obviously for the sake of a stream. Perceptual gain will not fill a worse fact.
• Check: ask “where was this faster than not streaming.” If the answer is “the whole run finished faster” and the wall clock did not, copy is lying. Then sit two records of equal total time, one streamed, and see whether the felt gap comes only from first token.

Related

• Same group: L1.12.1 Generation latency sits in the felt-wait band · L1.12.3 Token-by-token invites judging before completion · L1.12.4 Mid-stream self-correction shows overturned content · L1.12.5 Long background jobs should not stream
• Nearby: L3.06 Streaming presentation · L3.11 Streaming the generation process · I2.07 Perceived performance
• Search terms: time to first token · perceived speed · streaming vs duration