Completion time is meaningful only on successful tasks

What it is

Task completion time is the duration from start to a defined endpoint. Failures, abandonments, and timeouts have no such endpoint: the clock stops at a moderator intervention, a quit, or a cutoff, and that reading is truncated non-completion, not a slow finish. Mixing those times with successful ones averages different events. As an efficiency measure, time is defined only on attempts that actually completed.

Why it happens

Completion time is anchored on an end event. Without the anchor, the observation is censored or truncated: someone who takes a wrong turn and stops at two minutes, and someone cut off at a twelve-minute cap, both yield readings smaller than the time they would have used if they had continued, yet those readings are treated as short or long completions. Mixing in failures can also make a design that fails quickly look faster and a design in which people eventually find the path look slower. Keeping only successful times is a sample selected on success: on a hard interface the remaining people may be the more skilled, and their times do not represent the whole. Success rate and conditional time therefore have to travel together; conditional time alone hides who was screened out.

Studying it

Score success first, then compute time on the successful subset. Report failures and timeouts by count, stop reason, and stop time; survival models can use the censoring, but those values do not enter a mean completion time. When comparing designs, read success rate and time-among-succeeders together; if a variant has lower success and faster succeeders, check whether slower people were recoded as failures. Preregister the start and end events (first actionable screen to success state) so sessions are not timed from task reading in one case and from first click in another.

Where it stops holding

Practice or familiarization times usually stay out of efficiency claims. For expert routine work, failures are rare and conditioning barely changes the sample; for novices and hard tasks, it rewrites who remains. If the question is “how long until stopping,” call it attempt duration or time-to-abandon, not completion time. Field logs that never observe a success state can only yield proxy durations, and the end event must be named.

Applying it

• Score time only on attempts marked successful; leave failure rows blank or labeled incomplete—do not paste the stop time in as completion.
• Report “success rate + time among succeeders” as a pair; never send a mean duration alone.
• If succeeders got faster but far fewer people succeeded, find who was dropped before claiming an efficiency win.
• Time from the participant’s first action to the success state; exclude task reading and off-task talk.

Related

• Same group: Q3.07.2 Time distributions are usually right-skewed, so the mean misleads · Q3.07.3 Faster is not necessarily better without correctness
• Adjacent: Q3.06 Task success rate · Q3.08 Error rate and help-seeking
• Search terms: success-conditioned completion time · censored task time · time on task