A single item’s measurement error cannot be read from internal consistency

What it is

A multi-item scale can use covariation among items to estimate internal consistency, which lower-bounds measurement error. A single item is one observation with no parallel parts, so alpha, split-half, and other coefficients that need inter-item correlation cannot be computed. How much of the score is a stable construct, and how much is wording, mood, and a momentary reading, is invisible in that one response. The item is not therefore less accurate. The usual reliability paperwork simply has a blind spot.

Why it happens

Classical measurement writes an observation as true score plus error. With several items, if errors are roughly independent and true scores are shared, inter-item correlations rise and a consistency coefficient follows. A single item has no second item to compare against, so error and true score are bound in the same number. Retesting can estimate stability over time and also counts real attitude change as “error.” Correlation with a criterion can show convergence and also mixes the criterion’s own error and any mismatch of constructs. Missing internal consistency does not mean missing error; it means one way of pulling error out of a total is gone. Treating a single item as “cleaner” usually treats invisible error as zero.

Studying it

If a single item must be used, plan substitute evidence for error in advance: short-interval retest, correlation with a validated multi-item scale, known-group differences. When reporting those coefficients, say what each one confounds. Do not compute or claim alpha for a single item. A subsample can retake the construct with multiple items to calibrate attenuation in the main sample’s single item. Simulation or disattenuation needs an error estimate first; without one, do not publish a “corrected true correlation.”

Where it stops holding

Extremely specific, just-happened facts (“did you finish that task”) have a different error structure, and internal consistency was never the right tool. A continuous slider is still one observation; a nicer control does not grant inter-item reliability. Repeating the same sentence twice can produce a correlation, but that estimates reading stability, not multi-indicator reliability of a construct, and it annoys careful respondents.

Applying it

• In the metric table, mark single items as “no internal-consistency estimate” and list the substitute evidence that will be used.
• Do not justify the method with “only one item, so it is purer”; rewrite what error evidence the claim needs.
• For a single item used as a gate or a target, run at least a short retest or a multi-item criterion check; if it misses the pre-registered bar, demote it to exploratory.
• Check: if retest, criterion, or known-group evidence cannot be produced, the item does not enter the primary outcome.

Related

• Same group: Q3.16.2 Short scales trade dimensional resolution for lower burden · Q3.16.3 Short-scale psychometrics must be revalidated in the target population · Q3.16.4 A single rating cannot locate which process step failed
• Adjacent: Q3.15 Questionnaires and rating scales · Q3.02 System Usability Scale
• Search terms: single-item measure · internal consistency · attenuation