When options differ too little, comparison collapses into picking noise

What it is

Two abstracts sit side by side, almost the same claim. The pick is the right-hand one — “it looks a bit longer.” Length is not the task dimension; it is residual still visible. Noise-picking means that when the difference on the decision dimension sits below a discriminable threshold, relative evaluation does not stop. It switches to whatever still splits: word count, punctuation, the side seen first. The comparison process is still running; the information is already gone.

Whether the set counts as multiple options is distinctness as an entry condition. This entry governs what happens during comparison: below-threshold difference corrodes evaluation into grabbing noise.

Why it happens

Relative judgment is forced to completion. The interface asked for a pick; a pick will be handed in. Below the difference threshold, signal on the decision dimension cannot enter the judgment; residuals and position become the only input. People can still give reasons; those reasons are amplified noise and will flip on a retest.

Worse, confidence need not fall. A choice was made, a reason was spoken; the experience is of a successful comparison. Noise-picking is therefore hard to catch by introspection.

Studying it

Build pairs: decision-dimension difference above threshold, on threshold, far below (near-duplicates). Forced choice side by side; retest after a delay. Dependent variables: retest agreement, whether reasons land on the task dimension, confidence. Independent variables: embedding distance, human discriminability ratings, whether “equally good” is offered.

“Equally good / cannot tell” must exist as an option. Without that exit, every below-threshold condition is forced to a vote, and noise-picking is an experimental artefact.

Where it stops holding

When the user wants a tweak, a below-threshold difference is the goal; picking “a bit shorter” is not noise. In aesthetic tasks a tiny difference sometimes is the style. Near-duplicates of a closed answer are not noise-picking; they are a false choice and should merge. A screen reader flattens tiny visual splits, so there are fewer noise channels and people may say “no difference” sooner — a more honest failure. This entry does not treat count producing regret, nor left-side bias.

Applying it

• Before juxtaposition, test difference on the decision dimension; below discriminable, merge or redraw. Do not hand near-duplicates to comparison.
• Offer “much the same / cannot tell” as a legal result; do not force a click on one.
• If the only reasons given are word count, punctuation, or left/right, mark the comparison invalid; do not log it as preference data.
• Check: juxtapose a near-duplicate pair and count how many people still pick one and give a non-task reason. Those picks are noise. Then offer “cannot tell”; valid picks should drop.

Related

• Same group: L3.07.1 Juxtaposition drops the evaluation load from good-or-bad to which-is-better · L3.07.3 More options bring choice difficulty and decision regret · L3.07.4 People favour the first and left positions; order must be controlled or randomised · L3.07.5 Multiple options expose the system’s instability — a cost and a form of honesty
• Nearby: L3.01 Generating Multiple Options · L1.01 Mismatch Between Stochastic Output and Deterministic UI
• Search terms: noise-picking · just noticeable difference · indiscriminable alternatives