Novelty usually rises then falls; learning usually falls then rises
Aliases: non-monotonic treatment effect · novelty curve · learning curve
What it is
Two common post-release trajectories point opposite ways. Novelty temporarily lifts attention and use, then settles at whatever the value supports: rise then fall. Learning makes skilled users clumsy first and then recover with practice: fall then rise. Reading only the endpoint or only day one swaps the stories: novelty’s fade is taken for product failure, learning’s trough for a worse design, learning’s recovery for novelty that has not worn off.
Why it happens
Novelty is stimulus-driven: unseen objects produce exploration events; when the stimulus fades the events disappear and the curve drops from a peak. Learning is skill-interruption-driven: broken automatisms raise errors and time; repeated exposure rebuilds automatisms and the curve climbs from a trough. Both can sit on one aggregate series, peak cancelling trough, looking like “nothing happened.” Direction is a diagnostic cue, not a law: novelty that takes the form of avoidance (it looks messy, so people use it less) can fall first; learning that takes the form of discovering a hidden capability can rise first. The default expectation remains this opposed pair; departures need an extra mechanism.
Studying it
Plot the grouped time series with release as the origin and read the sign of the early slope and whether it reverses. Novelty-dominant series start positive then turn negative, often in lockstep with prompts, badges, and promotion. Learning-dominant series start negative then turn positive, in lockstep with repeat-use counts. Describe the shape with turning time, peak or trough depth, and where the recovered plateau sits relative to the pre-release baseline—not with a single slope. If only half a shape is observed (rise without fall, fall without rise), do not invent the missing half to complete a narrative.
Where it stops holding
Shape is a heuristic. Outages, price, and competitors can manufacture the same rise or fall. Metrics near a ceiling (success already almost perfect) flatten rises; metrics near a floor flatten falls. Different metrics can run opposite shapes at once—clicks rise then fall while completion time falls then rises—in which case there is not one “true” curve but two mechanisms in parallel. Without enough repeat use, learning never produces its second-half rise and the series stalls in the trough; a stall is not novelty.
Applying it
- Label post-release readings by shape: rise-then-fall is handled as novelty, fall-then-rise as learning. Do not extrapolate the endpoint from day one’s sign.
- Put observation points on the expected peak and the expected trough, and write what kind of reversal is required before a stable judgment.
- When a decision is demanded before any reversal, extend observation rather than writing a half-shape as the final effect.
- Check: split the post-release series into early and late slopes. If the signs oppose, the report must carry both ends and may not quote only one.
Related
- Same group: Q3.14.1 Early metric movement after a change can be transitory · Q3.14.2 Learning costs temporarily suppress metrics for existing users · Q3.14.3 Observation windows must be long enough · Q3.14.5 Segmented comparison is required to tell them apart · Q3.14.6 Engagement metrics pick up novelty more than task success · Q3.14.7 Major redesigns void historical baselines
- Adjacent: Q3.12 Funnel and retention analysis · Q3.04 A/B testing
- Search terms:
novelty curve·learning curve·non-monotonic treatment effect
Cards in the same group
- Q3.14.1Metric movement right after a change may not last
- Q3.14.2Relearning costs for existing users pull metrics down for a while
- Q3.14.3The observation window has to outlast the transient
- Q3.14.5Telling novelty from learning takes a split by people; the blended curve confuses them
- Q3.14.6Engagement metrics absorb novelty more readily than task success does
- Q3.14.7After a major redesign the old baseline is gone; a new comparison period has to be built