A one-tailed test lowers the significance bar and must be declared before the experiment

What it is

A one-tailed (one-sided) test puts the entire significance level into a side named in advance. At the usual α = 0.05, the critical value drops from about 1.96 on two tails to about 1.64 on one, so the same point estimate crosses “significant” more easily. The bar is lower because the other side was declared not worth rejecting, not because the evidence got stronger. Direction and tail count must be written into the plan before the data are seen; switching to one tail after the sign appears is moving the critical value after the fact. Product experiments usually need to see harm as well as gain; two-tailed is the default, not a taste for caution.

Why it happens

Two tails split the rejection region; one tail stacks the same area on one side, so the cutoff moves toward zero. If the true direction is the opposite of the declaration, a one-tailed test calls an extreme result on the other side “not significant” or does not test it, and a harmful move becomes harder to catch formally. Choosing the tail after seeing the data writes the realized sign into the rejection rule: positive, test the positive side; negative, the negative side—nominal α used twice. A pre-registered one-tailed test is defensible only when the direction has a strong prior and the opposite side is substantively defined as “same as zero, or not a claim.” Interface changes rarely meet that: copy, layout, and defaults can hurt someone.

Studying it

The plan states one tail or two, which side, α, and how the opposite direction will be reported (estimate and interval still shown, or pre-declared as not a claim). The registration timestamp must precede the outcome window. Analysis follows the registration; a wish to change tail count mid-flight is a new study, not an edit to the running test. Report the two-tailed p alongside so readers can see how much the one tail “helped.” When reviewing a directional claim, check whether that direction appeared in the materials or only after the sign of the result.

Where it stops holding

Equivalence and non-inferiority tests are directional by construction and have their own one-sided or two-sided conventions; they are not the same move as turning an ordinary superiority test one-tailed to clear a line. In safety or regulatory settings where the only question is “did it get worse,” a one-tailed test or a one-sided confidence bound can be the right tool and still must name the side beforehand. Exploratory data may be inspected for direction; that stage does not get one-tailed significance language. If n was planned under one tail and the actual test is two-tailed, power is systematically short.

Applying it

• Default the experiment document to two tails. A one-tailed test needs direction, rationale, and the opposite-side reporting rule written before the split.
• Do not tick “one-tailed” in the results meeting after the arrow is visible.
• If an external conclusion used one tail, the footnote must say how the critical value moved; if that move cannot be stated, rerun two-tailed before publishing.
• Check: recompute the same data two-tailed. If only the one-tailed test clears the line, do not write the result as a confirmed improvement.

Related

• Same group: Q3.21.1 A p-value is compatibility, not the chance the effect is real · Q3.21.2 After the fact you cannot tell whether n was enough · Q3.21.4 Width of the interval tells precision better than whether it includes zero
• Adjacent: Q3.13 Significance and practical importance · Q3.22 Multiple comparisons and result picking
• Search terms: one-tailed test · one-sided test · pre-registration