Mean completion time alone hides speed-accuracy differences

What it is

The mean completion time limitation is that comparing pointing devices or interfaces only by mean time hides different speed-accuracy strategies. Shorter time may reflect faster, equally accurate control, or early release, more misses, more undo, or corrections excluded from timing; time alone cannot distinguish these cases.

Why it happens

A target-acquisition task lets people freely pick a working point along a speed-accuracy tradeoff (SAT) curve: further toward the fast end, endpoint error grows; further toward the accurate end, time grows. A single mean completion time from one experiment is just the point on that curve where participants happened to settle that day, not the curve itself. If participants in two conditions — say two devices — happen to settle at different points on their respective curves, one biased toward speed and the other toward stability, comparing that one time difference conflates "participants chose a different strategy" with "the device has a different capability" — the two are folded into a single number and cannot be pulled apart afterward. Biased error handling compounds the problem: quietly dropping failed or retried trials keeps only the samples that happened to be both fast and accurate, making the faster-looking condition seem better than it really is.

Studying it

For every condition, report mean or full-distribution completion time, error rate, retry count, endpoint deviation, and throughput together, and state exactly where timing starts and stops, how errors are handled, and whether retries count toward total time. A more rigorous approach actively measures multiple points under multiple accuracy demands to plot a full speed-accuracy curve, or at minimum compares time only after calibrating error rates to be equal across conditions in a pilot run (adjusting target width until both conditions produce a similar error rate) — this is an isoperformance comparison, and only when error rates are matched does a time difference cleanly correspond to a speed difference.

Where it stops holding

Time remains a legitimate primary metric when a pilot has already calibrated error rates to be comparable across conditions, or when the task genuinely cares only about speed with minimal accuracy demands. But an overall error rate showing "no significant difference" does not mean the problem is gone: a small number of severe errors — triggering an irreversible action by mistake, say — can matter far more than the time difference even at a low rate, and distinct user groups may each stably settle at different points on the curve, a systematic pattern that an aggregate mean erases. Continuous tasks such as dragging or drawing also need path quality and in-process stability, since reaching the same endpoint does not mean the movement itself was equally well controlled.

Related

• Same group: C1.15.1 Throughput captures a pointing device's speed-accuracy tradeoff, not speed alone · C1.15.2 Fitts' law models movement time as a logarithmic relation to target distance and width · C1.15.3 Effective width is recalculated from endpoint distribution rather than nominal width · C1.15.4 Throughput across tasks and postures cannot be compared directly
• Nearby: Q1 Research methods and evaluation · C1.03 Control-display gain
• Search terms: speed-accuracy tradeoff · error rate · throughput