After-the-fact trace needs input, system version, and output kept together

What it is

After something goes wrong you have to ask what was actually given, which build ran, which passage came out. Miss one of the three and the story will not close: input without output knows only the request; output without version cannot tell whether a already-fixed model did it; version without complete input (system prompt dropped) cannot be saved by a seed. The trace triad must be one atomic record of a generation, not three logs glued later.

A seed is extra. The triad is the floor.

Why it happens

Distributed systems drop the three pieces on client, gateway, and inference. Retention clocks differ; glued together a piece is often missing. Legal and incident review want that world, not “a recent similar request.” Without output, review can only replay, and replay under sampling is not that instance. Without version, you cannot tell whether a current fix covers then. Without complete input, even replay has no premise.

The user-visible sentence is never the complete input. Tool lists, retrieved passages, hidden system prompts are in the condition. “Input” in the triad must be what the model actually ate, plus what the user thought they submitted — both — or you cannot explain “I did not write that, why did it answer.”

Studying it

Run a controlled incident, then trace from current logs. Record how often each piece of the triad is missing, time to assemble a complete story, how often the story can only approximate by replay. Independent variables: whether an atomic record is written at generation-complete, whether retention clocks align. Dependent variables: story completeness, rate of taking another draw as then.

Aligned retention is the crux. Output kept 7 days, version 90, and day 8’s trace must miss.

Where it stops holding

Throwaway, no-consequence play may skip storing output, and then also skip promising after-the-fact explanation or appeal. Inputs the law requires deleted must take output and the version key with them, or anonymise; otherwise the triad becomes a residual identifier. Partial output from an aborted stream is still output, stored as what was on screen. This entry does not treat using many draws instead of one comparison.

Applying it

• At every generation-complete (including a user-visible abort) write an immutable record: user input, hash of the complete model input and an authorised copy, model and tool versions, output. The three share a lifetime.
• Tickets and timelines link that record, not “run this sentence again.”
• On user export or appeal the triad travels together; if a corner is already gone, say which.
• Check: pick a conversation from last week at random and ask engineering to produce, without replay, the words then on screen, the version then, the complete prompt then. Missing one, trace is guessing. Then stagger the three logs’ retention by a day and see which day the story breaks first.

Related

• Same group: L1.11.1 Stochastic defects will not replay · L1.11.2 Seeds do not survive version changes · L1.11.3 Non-reproducibility flattens the learning curve · L1.11.5 One draw cannot rank two systems
• Nearby: L4.15 Accountability and traceability · L1.06 Graceful degradation of AI failure · L3.13 User feedback loops on generation quality
• Search terms: input-version-output triad · atomic generation record · after-the-fact trace