Without a control group, treatment cannot be separated from the time trend

What it is

A before–after look at the same cohort or the same product can move because of the treatment, or because of a time trend that would have happened anyway: season, marketing cadence, weekday, maturation, a news event. A control group is a concurrent arm that does not receive the treatment and is otherwise as comparable as the design can make it, used to estimate what this period would have done on its own. Without that arm, treatment and time sit inside one difference and cannot be pulled apart. This is not the claim that an attitude score is a lagged snapshot; it is a missing concurrent contrast in the experimental structure.

Why it happens

The before–after gap equals treatment plus history plus maturation plus measurement drift. Holidays, ads, a competitor outage, a silent client update will move conversion, time, and ratings with no interface change. The eye assigns the nearest launch as the cause because it is near in time, not because other paths have been blocked. A concurrent control subtracts shared shocks: if both arms are lifted by the same season, the between-arm gap can still point at treatment. The control has to be truly concurrent and uncontaminated. Substituting “last month” or “this week last year” treats another time slice as a control; season and calendar events are not cancelled.

Studying it

State in the plan who the control arm is, when it starts, and how treatment is kept out. To test whether a historical control would have been enough, pretend the treatment occurred in several pre-launch windows and see whether the fake treatment also “hits”—if it does, time trend can already write the same story. An interrupted series needs a pre-registered break and trend model, plus a control series; one own-curve remains weak. A staggered rollout lets units not yet reached act as a rolling control; still check that wave is not entangled with region or channel. The primary estimate is the concurrent contrast; the before–after gap is description.

Where it stops holding

When treatment must cover everyone (mandated compliance, a full safety patch) and no arm can be held back, causal claims step down: use several pre-trend diagnostics and an external series, and say the identification is weak. A control that is quickly contaminated—control users see the treatment, shared stock is drained by the treated arm—stops being a control. When the effect is a change in time structure (flattening a peak), the control is still useful, but the comparison is curve shape rather than a single-window mean. In a very short window the trend may be small; say why it can be ignored rather than ignoring it by default.

Applying it

• Default a launch comparison to two concurrent arms. If no control arm can be named, the conclusion may say “observed after treatment,” not “treatment caused.”
• Do not replace a control with “versus last week.” If history is all that exists, run fake-launch tests in prior windows and show them.
• Mark known calendar events (promos, releases, incidents) on one timeline; both arms must share those marks.
• Check: hide the treatment date and ask a colleague who was not there to name the cause from a single curve. If several equally plausible events appear, do not ship a causal sentence.

Related

• Same group: Q3.18.2 Randomize at the unit where the effect occurs · Q3.18.3 Within-subjects designs need order handled by counterbalancing · Q3.18.4 A group gap is a treatment effect only after other variables are controlled
• Adjacent: Q3.04 A/B testing · Q3.14 Novelty and learning effects
• Search terms: control group · time trend · difference in differences