Structured output passes through temporarily illegal intermediate states while streaming

What it is

JSON is still coming out; a brace has not closed; a downstream form parser already eats it as “a structure,” throws, and the UI goes red with failure. Two seconds later the brace lands and the object would have been legal. Illegal intermediate states means structured output must pass through prefixes a parser will not accept; sending a prefix into downstream as a finished artefact is a protocol error, not a model that cannot write.

Whether it looks like a final draft is for human eyes. The recipients here are parsers, and people tripped by parsers.

Why it happens

Autoregression grows by tokens; a legal structure grows by a closed tree. Any true prefix may, before closure, fail to be legal JSON, compilable code, or an aligned table. If streaming broadcasts every prefix on the same “result” channel, downstream has no type for “still growing” and can only treat it as a damaged finished object. People see red, read a temporary syntax failure as a task failure, discard or change the prompt early, and cut a generation that would have succeeded.

Incremental parse, length prefixes, schema-then-values, all give the intermediate a legal type. With no type, the intermediate can only be an error.

Studying it

For JSON / table / code, compare: every prefix sent to a parser versus send only at closure or checkpoints versus an incremental parser. Dependent variables: false failure counts, rate of aborting from a false failure, legal rate of the final. Independent variables: depth of structure, whether the UI displays parse errors.

Split “intermediate parse failure” from “final illegal.” The former is an expected event under streaming.

Where it stops holding

Plain prose has no parser; the intermediate problem is weak and only the human-facing incomplete state remains. When users only want to read and will not paste a half-slice into a compiler, the raw prefix may be shown, but do not offer “insert as JSON.” If a tool-call protocol specifies incremental events, use them; do not fake a complete payload. When abort lands before closure, ownership of the half-structure is the next layer.

Applying it

• Do not let structured results enter a parsing channel before closure; what humans see in flight is a read-only prefix, the action “wait until the structure is complete.”
• If incremental consume is required, use an incremental parser or checkpoints (each complete element), not a one-shot parse.
• Red text on an intermediate should read “not yet closed,” not “generation failed.”
• Check: send half JSON into the product’s actual downstream. If downstream reports failure and people stop because of it, the intermediate was treated as terminal. Send after closure; the same downstream should accept.

Related

• Same group: L3.11.1 The value of streaming is an early signal one can judge, not a shorter total duration · L3.11.2 People can abort from early content; abort rights are streaming’s main gain · L3.11.3 Incomplete and complete content must be visually distinguishable · L3.11.5 After abort, ownership of what was already generated must be clear: usable, redo, or void
• Nearby: L3.06 Streaming Presentation · L3.12 Editing and Taking Over Generated Content
• Search terms: partial JSON · incremental parse · illegal intermediate state