Time distributions are usually right-skewed, so the mean misleads

What it is

Even on successful attempts, completion times rarely form a symmetric bell. Perception, movement, and system response impose a floor, while one lost search, one recovery, or one lapse can stretch the ceiling, so the distribution is right-skewed. The arithmetic mean is pulled by a few long tails and often represents neither typical experience nor a stable two-group contrast. A median, a quantile, or a location on a log scale is closer to “how fast a typical completion is.”

Why it happens

A task is a chain of steps; delay in any step adds to the sum, and step times themselves tend to be skewed. Failed search, error recovery, and network waits create a long right tail; there is no matching “especially fast” tail, because speed cannot undercut physiological and interface floors. The mean then follows the unluckiest trials: eight people at forty seconds and one at eight minutes already yield a mean unlike those eight. In a small sample, which group that one person falls into can reverse “which design is faster,” a flip that has nothing to do with interaction quality.

Studying it

Plot a histogram or empirical distribution to see skew and outliers before choosing a summary. Common practice is to report the median and interquartile range, or to estimate location after a log transform; a geometric mean is also used for strictly positive times. Inference should be robust to skew or use permutation or bootstrap methods, rather than sending raw seconds into a default two-group t-test. Long-tail cases should be reviewed on video to separate interface-caused recovery from off-task interruption; the latter can be dropped by a prespecified rule, the former should remain part of the distribution. When comparing designs, ask whether the whole distribution shifted left, not only whether means differ.

Where it stops holding

Highly practiced, fixed-sequence operations can be nearly symmetric, and the mean then does less damage. If the question is the worst case—timeout risk, peak wait—the right tail is the target, and compressing it into a median hides risk. On tiny samples the median is unstable too; show every point. Deleting every long tail as an outlier deletes the completions that most expose problems.

Applying it

• Default to median plus interquartile range for completion time; if a mean appears, accompany it with the shape of the distribution.
• Do not use “faster by N seconds on average” as a ship argument before looking at the histogram.
• Replay sessions beyond a prespecified quantile and mark interface recovery versus off-task interruption.
• Write the acceptance rule as “median down and upper quartile not worse,” so one long tail cannot kidnap the mean.

Related

• Same group: Q3.07.1 Completion time is meaningful only on successful tasks · Q3.07.3 Faster is not necessarily better without correctness
• Adjacent: Q3.13 Statistical significance and practical importance · Q3.06 Task success rate
• Search terms: right-skewed task-time distribution · geometric mean · time-on-task median