Q3.14.4Novelty and learning produce opposite metric trajectoriesdesignresearch

Novelty usually rises then falls; learning usually falls then rises

Aliases: non-monotonic treatment effect · novelty curve · learning curve

What it is

Two common post-release trajectories point opposite ways. Novelty temporarily lifts attention and use, then settles at whatever the value supports: rise then fall. Learning makes skilled users clumsy first and then recover with practice: fall then rise. Reading only the endpoint or only day one swaps the stories: novelty’s fade is taken for product failure, learning’s trough for a worse design, learning’s recovery for novelty that has not worn off.

Why it happens

Novelty is stimulus-driven: unseen objects produce exploration events; when the stimulus fades the events disappear and the curve drops from a peak. Learning is skill-interruption-driven: broken automatisms raise errors and time; repeated exposure rebuilds automatisms and the curve climbs from a trough. Both can sit on one aggregate series, peak cancelling trough, looking like “nothing happened.” Direction is a diagnostic cue, not a law: novelty that takes the form of avoidance (it looks messy, so people use it less) can fall first; learning that takes the form of discovering a hidden capability can rise first. The default expectation remains this opposed pair; departures need an extra mechanism.

Studying it

Plot the grouped time series with release as the origin and read the sign of the early slope and whether it reverses. Novelty-dominant series start positive then turn negative, often in lockstep with prompts, badges, and promotion. Learning-dominant series start negative then turn positive, in lockstep with repeat-use counts. Describe the shape with turning time, peak or trough depth, and where the recovered plateau sits relative to the pre-release baseline—not with a single slope. If only half a shape is observed (rise without fall, fall without rise), do not invent the missing half to complete a narrative.

Where it stops holding

Shape is a heuristic. Outages, price, and competitors can manufacture the same rise or fall. Metrics near a ceiling (success already almost perfect) flatten rises; metrics near a floor flatten falls. Different metrics can run opposite shapes at once—clicks rise then fall while completion time falls then rises—in which case there is not one “true” curve but two mechanisms in parallel. Without enough repeat use, learning never produces its second-half rise and the series stalls in the trough; a stall is not novelty.

Applying it

  • Label post-release readings by shape: rise-then-fall is handled as novelty, fall-then-rise as learning. Do not extrapolate the endpoint from day one’s sign.
  • Put observation points on the expected peak and the expected trough, and write what kind of reversal is required before a stable judgment.
  • When a decision is demanded before any reversal, extend observation rather than writing a half-shape as the final effect.
  • Check: split the post-release series into early and late slopes. If the signs oppose, the report must carry both ends and may not quote only one.

Related

  • Same group: Q3.14.1 Early metric movement after a change can be transitory · Q3.14.2 Learning costs temporarily suppress metrics for existing users · Q3.14.3 Observation windows must be long enough · Q3.14.5 Segmented comparison is required to tell them apart · Q3.14.6 Engagement metrics pick up novelty more than task success · Q3.14.7 Major redesigns void historical baselines
  • Adjacent: Q3.12 Funnel and retention analysis · Q3.04 A/B testing
  • Search terms: novelty curve · learning curve · non-monotonic treatment effect

Cards in the same group

Quick Actions

Share

Share this page

ios_share

https://hci.top/en/handbook/Q3.14.4