Q3.11.3Non-retrospective event capturedesignresearch

Missing events cannot be backfilled after the fact

Aliases: no retroactive logging · instrumentation lag · cannot backfill events

What it is

Clients and servers emit a record only at the moment an event fires. An undeclared event has no row in historical sessions. Adding the event later accumulates from the patched version forward; it cannot make last month’s users “click again.” A proxy stitched from neighboring events is a measurement of something else, not the missing event recovered. Time in logs runs one way.

Why it happens

A log is a forward-written stream. There is no negative-time insert on disk: a phone does not keep an unsent spare “filter applied” event. If the server did not persist a field from the request body, the raw packet has usually already been dropped under a retention rule. Inferring “compared two products” from page views, click coordinates, or screenshots changes the construct: comparison is a cognitive act, browsing is another kind of trace. A proxy can be a stopgap, but its error and bias belong to the new metric and cannot be spliced onto an old trend as if the series were continuous.

Studying it

Mark the earliest answerable date for each critical question: the date the corresponding event reached production in full. Research that needs historical depth first checks whether an acceptable proxy exists before that date, and writes down how the proxy’s definition differs from the target event. After a patch, run old and new in parallel for a while to estimate proxy bias, rather than grafting the new event onto the old series. If the decision window already sits in an invisible past, switch to interviews, support records, or a new prospective capture, and narrow the temporal claim.

Where it stops holding

When raw requests still sit in a warehouse, or a device still holds unsampled debug logs, a limited field can sometimes be replayed—that is residual raw material, not an event created from nothing. Compliance deletion and expiry close that door. Interactions that never left the client cannot be rescued by any backend. Real-time systems may retransmit from a short buffer, but the buffer is seconds or minutes and does not help a quarterly review.

Applying it

  • Footnote dashboards with “this number exists from date D”; leave earlier cells empty rather than filling them with a grey estimate.
  • In review, ask “if we only think of this question next week, is the history already gone”; if yes, ship the event with the feature, not after.
  • When a historical stand-in is unavoidable, name the proxy separately; do not reuse the new event’s name.
  • For regulatory or ledger questions, the event must exist on the day of release; data patched later does not count as contemporaneous evidence in an audit.

Related

  • Same group: Q3.11.1 Logs record behavior but not motive · Q3.11.2 Instrumentation design determines which questions can later be answered · Q3.11.4 Inconsistent event names prevent joining data across versions · Q3.11.5 Sampling and dropped reports systematically undercount rare actions · Q3.11.6 Ad blockers and privacy settings permanently omit some users · Q3.11.7 The same event may fire under unequal conditions across platforms
  • Adjacent: Q6.02 Goals–signals–metrics · Q1.16 Research time and budget constraints
  • Search terms: non-retrospective event capture · backfill logging · instrumentation lag

Cards in the same group

Quick Actions

Share

Share this page

ios_share

https://hci.top/en/handbook/Q3.11.3