Fairness goals must be chosen explicitly before design

What it is

A fairness specification treats fairness as an acceptance constraint on par with latency budgets and accuracy targets, written before system design: which attributes are protected, which statistical criterion applies, the tolerated deviation, the subgroup reporting plan, and which criteria were rejected and why. Its opposite is "deal with it after launch" — the model is trained, an incident lands, and someone picks from the menu of definitions whichever one the system already nearly passes and declares compliance.

Why it happens

A definition chosen after the fact cannot constrain what has already hardened. The choice of fairness criterion determines how data is collected (subgroup evaluation needs adequate per-group samples, which cannot be retrofitted), how labels are defined (a "success" label inheriting historical discrimination will merely redistribute harm under any criterion), and how thresholds are set (error-rate balance may require per-group thresholds, impossible in a single-threshold pipeline). The deeper reason is incentive structure: when selection happens after results are known, people slide toward the criterion their system passes, and evaluation decays from finding problems into endorsing decisions. Moving the choice before results exist confiscates the adjustable die — the same logic that applies to every acceptance spec (performance, safety).

Where it stops holding

Choosing ex ante does not mean choosing forever: user-mix drift and shifting base rates invalidate the original criterion, so the spec needs revision triggers rather than permanence. During exploration, before the decision semantics are even stable, a full spec becomes ritual; a minimal tier ("protected attribute list + no automated decisions yet") suffices. And a spec cannot constrain goals that live outside it — if growth metrics outweigh fairness in performance reviews, the document loses however complete it is; its force comes from binding to release gates, not from existing.

Applying it

• Complete a one-page fairness spec before model development starts: protected attributes and their justification, the chosen criterion, tolerance thresholds, the subgroup evaluation plan, and rejected criteria with reasons, co-signed by product and legal.
• Turn the spec into acceptance items: a post-training subgroup metrics table that misses tolerance blocks release, with the same force as an accuracy miss.
• Attach revision triggers to every clause: new-market launch, significant user-mix change, upstream data-source swap.
• Verify: audit the most recent launch decision — if metrics for the rejected criteria were never computed, the spec was bypassed; trace why (data never collected, metric never built) and close the hole.

Related

• Same group: P4.02.1 Different definitions of fairness conflict mathematically · P4.02.2 Group and individual fairness can be incompatible
• Adjacent: P4.14 Institutionalizing ethics review · P4.09 Algorithmic fairness and disparate impact
• Search terms: fairness specification · fairness requirements · model card