Differences in wording produce differences in results

What it is

“Make this shorter” and “compress to one hundred twenty words, keep numbers and proper names” point at one intent and often walk out as two artifacts. Prompt sensitivity means that in open input, surface wording is a real independent variable, not disposable wrapping. People treat near-synonyms as “close enough.” The decoder reweights on word choice, order, punctuation, and a missing constraint.

This is not “the user has not yet learned how to ask.” Two sentences can already be synonymous for the speaker and still fork. It is also not the later folk theory that a magic phrase must be memorized — only that the mapping itself is unstable.

Why it happens

Autoregressive models take the prompt as conditioning context. A near-synonym shift moves attention: one word pulls toward tone, another toward length, a third toward a tool schema. Unwritten constraints are not asked back; they are filled from the prior. Two “almost the same” sentences therefore sample different regions of the conditional. Temperature then scatters points inside a region, so even one sentence wobbles across runs — wording is an extra layer of input noise on top of that wobble.

Human language understanding is intentional: a listener collapses wording differences onto one goal. Fitted onto a generative entry, that listener model reads a result split as a random fault, not as “you changed the independent variable.”

Studying it

Freeze an intent spec (for coders, not for the model). Let different people each write one prompt, or batch paraphrases from a paraphraser, and run them under one decoding setup. Measure disagreement on the task metric, structural slots, and forbidden items. Independent variables: edit distance of the paraphrase, whether numeric constraints are explicit, prompt length. Dependent variables: pairwise output agreement, slot hit rate, human judgment of “still the same task.”

Separate sampling noise from wording: repeat each prompt n times, read within-prompt variance first, then between-prompt variance. Only if between stays larger than within does wording drive the result.

Where it stops holding

Calls under hard structure — a JSON schema, three enum fields, downstream rule checks — squeeze wording differences until they are almost invisible outside format. When temperature is locked and the system rewrites the user string into an internal canonical request, user-side wording is compiled away. In creative exploration the fork is the point, not a defect. This entry covers task generation where the user thinks they said one thing and the system treated two.

Applying it

• Before send, parse the user’s words into a visible constraint list (length, object, bans). Let people edit the list rather than gamble on another synonym.
• When offering “try another wording” on a prompt that already worked, keep both results side by side so the fork is visibly from phrasing, not a failed refresh.
• When stability matters, do not send the raw wording of the same intent to the model every time; normalize to an internal task representation first.
• Check: three requests humans rate as synonymous, one generation each on the same account and settings (repeat each once more to estimate sampling noise). If task-level differences among the three stay larger than each sentence’s internal scatter, the UI still treats wording as wrapping.

Related

• Same group: L2.01.1 Open input does not cue the range of capability · L2.01.2 Not knowing how to say it is the main barrier · L2.01.4 An empty box conveys no boundary; the first sentence is a guess · L2.01.5 Open input steers failure attribution toward “I said it badly” · L2.01.6 Synonymous phrasings yield different results, so users invent phrases to memorize · L2.01.7 Openness makes functions non-enumerable; the product can no longer show a complete capability list · L2.01.8 Error messages for open input stay vague because the system does not know what the user meant to do
• Nearby: L1.01 Mismatch Between Stochastic Output and Deterministic UI · L2.15 Ambiguity and Clarifying Follow-ups · L2.04 Complementarity of Parameter Controls and Natural Language
• Search terms: prompt sensitivity · paraphrase robustness · phrasing-dependent output