Case and diacritics affect collation results

What it is

Collation strength controls how fine a difference a string comparison observes. The Unicode Collation Algorithm normally compares primary weights first, then secondary, tertiary, and—when requested—higher levels. In many Latin-script tailorings, base-letter differences are primary, diacritics are secondary, and case is tertiary. Language tailorings and parameters can change that interpretation, so “primary always ignores accents” and “tertiary always means case” are not universal laws. Strength is a comparison policy, not an instruction to lowercase stored text or strip combining marks permanently.

Why it happens

Layered weights let a collator form broad reader-meaningful groups before resolving finer distinctions. Under an applicable configuration, a base letter sets the main position, an accent separates variants of that base, and case separates them last. Other languages promote an accented form to a distinct primary letter, and some tailorings—French is a familiar case—can compare secondary weights backwards. CLDR exposes strength alongside case level, case first, and alternate handling, so matching at the same nominal strength can still differ across configurations. A display list may retain fine distinctions for deterministic order; forgiving search may compare at a lower strength. Username uniqueness, database keys, and security-sensitive equality require separately specified normalization and exact comparison, not an accent- or case-ignoring collator used as identity.

Studying it

Construct minimal pairs that change only a base letter, diacritic, case, punctuation, or canonical encoding, then inspect equivalence classes and order at each strength and locale. This isolates the responsible level better than checking only a final list. For search, test exact input, omitted diacritics, and changed case; record recall, false matches, ambiguous result sets, and correction effort. For display order, examine whether a lower strength makes many names compare equal and audit the stable tie-breaker. A reproducible report states locale, collation type, strength, case and alternate options, normalization behavior, and library version; “case-insensitive” alone is underspecified.

Where it stops holding

Weight levels belong to a particular collation and are not a direct mirror of Unicode character categories. A mark can be a secondary distinction in one language while the resulting letter is primary in another; casing also varies by script, position, and locale. UCA requires canonically equivalent sequences to collate equally, which an implementation may guarantee through canonical normalization or an equivalent mechanism; compatibility characters, punctuation, numbers, and emoji involve additional policies. Lower strength expands equivalence classes and can improve discovery while conflating words users consider distinct. Higher strength still does not automatically implement the security semantics of login identifiers, domain names, file paths, or passwords.

Applying it

• Configure each task separately: use a linguistically appropriate, deterministic collator for visible order; choose an explainable tolerance for search; and use business-defined exact normalization plus stable IDs for uniqueness and identity.
• Set locale, collation type, strength, case level, case first, and punctuation handling explicitly. Do not imitate collation with blanket lowercasing or removal of combining marks; those operations mutate data and create invalid equivalences.
• If the selected strength makes several strings compare equal, finish display ordering with a higher-strength comparison or stable record ID. Pagination and caches must reuse the same comparison chain so equal groups do not drift across pages.
• Review minimal case and diacritic pairs, canonically equivalent encodings, and realistic proper names with speakers of the language. Validate visible order, search recall, and uniqueness conflicts independently.

Related

• Same group: S2.08.1 Alphabetical order varies by language · S2.08.2 Chinese can be sorted by pinyin, stroke count, or radical · S2.08.4 Index navigation must switch with language
• Adjacent: S2.05.5 Character-set restrictions reject valid names · S2.09.1 Regular-expression validation usually embeds a single-region assumption
• Search terms: collation strength · secondary weight accents · tertiary weight case