Derive metrics from goals, not from available data
Aliases: Goals-Signals-Metrics · top-down metrics · data-first metrics
What it is
Goal-derived metrics walk down from “what we need to judge has happened,” not up from fields already in the logs. The first hop in Goals–Signals–Metrics (GSM) is the goal: write what success looks like, then ask which phenomena would indicate it, and only then specify a counting rule. Starting from available data reverses the path: clickstreams, dwell time, and push reach sit in the warehouse, so they become “experience metrics.” Collectable is not the same as worth judging.
Why it happens
Instrumentation grows by engineering convenience. Events are added at development time, fields are kept or dropped by storage cost, and reports are built from ready-made aggregates. That supply chain never passes through a goal. Dashboards fill with numbers that answer outdated questions, while a real goal—such as “the on-call doctor can find allergy information within two minutes”—may have no field at all. Data-first also locks later design: because duration is collectable, optimization becomes lengthening sessions; because open rate is collectable, optimization becomes reddening the entry. Once the goal is postponed, the product is shaped by data supply rather than by the question. GSM reverses the derivation so missing observations surface: a clear goal that current data cannot meet should add collection or revise the goal, not switch to a field that happens to plot.
Studying it
Trace every metric on the live dashboard back to the goal it claims to serve; mark those with no goal as unowned. For each real goal, ask whether existing data could distinguish attainment from failure, and record the gap. Compare two generation processes: one writes goals then picks metrics, the other lists collectable fields then pastes on goals, and count how many of the latter goals were attached after the fact. Decision replay also works: of the metrics cited in a launch review, how many had a matching item on the pre-meeting goal list.
Where it stops holding
Early exploration may read existing logs to discover what the goal should be; that is question formation, not a measurement scheme. Engineering indicators for compliance, performance, and incidents need not derive from an experience goal, and they should not be renamed as experience success. Goal derivation does not make the goal itself correct; a wrong goal, rigorously decomposed, is precise misdirection. Platform limits may force a temporary weaker metric; label it as a proxy pending collection, rather than promoting the compromise into a principle.
Applying it
- Open the cycle with one sentence of the success to be judged; do not start the meeting with “what numbers do we have.”
- Under each goal, allow only metrics that serve it; if a ready-made field does not fit, record a gap instead of rewriting the goal to match the field.
- New events must carry the goal they serve; unowned events do not enter experience reports.
- Quarterly cleanup: keep metrics that point at a goal; retire those that point only at “because we can collect it.”