A usability test lasting minutes ends before arm fatigue has time to show up

What it is

A single task in a standard usability test usually lasts tens of seconds to a few minutes — a window shorter than the time scale on which arm-raise fatigue actually begins to show. The test comes back clean, and only after real-world deployment, once people use the interaction for extended periods, do complaints about sore arms start showing up. This is a systematic blind spot in the testing method, not a matter of the test being run carelessly.

Why it happens

Usability test tasks are usually designed around completion rate and task time as the core metrics, which only require the participant to perform an action once or a few times — not enough repetitions for accumulated load to cross the endurance threshold described elsewhere before the test ends. Gaps between test trials, participants' novelty-driven enthusiasm, and the tendency to tolerate discomfort while knowingly being observed all further suppress the likelihood of reporting discomfort during the test. The net effect is a systematic optimistic bias in usability judgments for arm-raise interactions, not random noise.

Studying it

Exposing this problem requires deliberately extending task duration or repetition count beyond the expected real-world session (having participants operate continuously for five to ten minutes instead of walking through the flow once), while inserting fatigue ratings or EMG recordings at fixed intervals throughout rather than asking for an overall impression only at the end. Diary studies or field observation can also help, by capturing the actual distribution of continuous-use durations in real deployments and working backward to the duration a test should cover.

Where it stops holding

This blind spot only appears when test duration is markedly shorter than actual continuous-use duration. If the target interaction is itself an occasional, few-seconds action (a one-off confirmation gesture, say), the standard short test doesn't have this problem, and extending the test brings no benefit.

Applying it

• At the test-planning stage, first estimate the interaction's typical continuous-use duration in the real deployment, and set the test task's duration or repetition count at least at that scale, instead of defaulting to a generic few-minute task template.
• Insert a brief fatigue rating at fixed intervals during the test (e.g., every 15–30 seconds) to build a curve over time, rather than asking once at the end.
• Verification: compare the fatigue-rating curve and rate of user-initiated stop requests between a "standard short task" and an "extended to real-use duration" version of the same interaction. If the extended condition surfaces stop requests or a rating spike the short task never showed, the original test plan has a blind spot, and the extended version needs to become part of the formal evaluation process.

Related

• Same group: A8.19.1 unsupported arm-raise fatigue sets in within tens of seconds · A8.19.2 fatigue accelerates jointly with elevation angle and duration · A8.19.3 perceived fatigue lags physiological fatigue · A8.19.5 high-frequency actions must not require overhead reach
• Nearby: A8.22 repetitive strain injury · A9.08 performance and secondary-task measurement of load
• Search terms: usability testing blind spot · prolonged interaction fatigue · ecological validity