Different definitions of fairness conflict mathematically
Aliases: fairness impossibility · incompatibility of fairness criteria · no universal fairness metric
What it is
"Fairness" in algorithmic contexts is not one concept but a family of distinct statistical definitions: calibration (same scores mean same observed outcome rates across groups), error-rate balance (equal false-positive and false-negative rates), equalized odds (errors unrelated to group membership given the true label), and demographic parity (equal positive-prediction rates). Impossibility of fairness results show that when group base rates differ and predictions are imperfect, several pairs of these definitions cannot hold simultaneously — satisfying one necessarily violates the other. This is not unfinished engineering; no solution satisfying all of them exists.
Why it happens
The conflict lives in the structure of conditional probabilities. Different base rates mean the same score carries different true meanings across groups: if you force equal score-to-outcome meaning (calibration), a high score in the higher-base-rate group must catch more true positives, so false-positive rates diverge; if you force equal false-positive rates, you must set different thresholds per group, and score meaning is no longer uniform. Only two degenerate cases escape: predictions perfect enough to eliminate error, or base rates already equal. Each definition encodes a moral stance — calibration suits risk communication (don't alarm low-risk people), error-rate balance suits punitive decisions (the cost of false accusations is shared across groups). Choosing a definition is choosing a stance for the product, not picking the "more correct" formula.
Studying it
The impossibility theorems are proven (for pairwise incompatibilities among calibration and error-balance criteria); researchers can derive which pairs of criteria can coexist in their own setting. The empirical route audits deployed systems: compute per-group error rates and test which criterion is violated and by how much; variables include the base-rate gap, model discrimination, and the decision threshold. Methodological caution: base rates are usually unobservable (no ground truth for true crime or true risk), so audits rely on proxy outcomes that may themselves be biased — distortions like "arrest records ≠ crime" amplify in cross-group comparison.
Where it stops holding
Impossibility holds under the standard setup: binary decisions at a single point, differing base rates, imperfect prediction. It does not apply when prediction quality is high enough that the conflict collapses (rare for commercial models); when decisions can route around scores into individualized procedures (human review, evidence-gathering), where statistical criteria give way to procedural design; or when per-group thresholds satisfy one criterion while creating a new dispute — "same score, different treatment" — which relocates rather than resolves the conflict.
Applying it
- At requirements time, write down the chosen criterion and the harm model it reflects: which side do the errors fall on, who bears them; pick calibration or error-rate balance accordingly.
- Record the rejected criteria and why, as part of the decision document; after launch, re-audit against the rejected criteria to confirm the cost is still acceptable.
- Before deployment, simulate with real group base rates whether the chosen criterion can even be approximately met on current data.
- Verify: produce a per-group score-to-outcome table; any column whose conditional rates deviate across groups beyond preset thresholds triggers review.