The same outcome has many counterfactuals; choosing which to show is itself a value judgement

What it is

This loan denial has at least three flip paths: raise income, change postcode, drop a credit card. All three go through on the model. The interface lighting only one is choosing a life on the user’s behalf. The same outcome has many counterfactuals; which to present is itself a value judgement.

The algorithm’s “nearest” is not neutral. Near is a kind of distance; the distance already buries what counts as cheap and what as dear.

Why it happens

Paths to the other side are a set, not a point. The generator has to pick in the set; the picking rule (L1, mutable fields, looking respectable) hides some people’s feasible roads and puts others on the table. The road put on the table is read as “what the system wants you to do.” If changing postcode is picked, users think the system is telling them to move, even though raising income would also suffice.

Wachter-style nearest neighbour, if Euclidean, lets numerically small edits win. Numerically small is not ethically cheap: changing a sensitive proxy may be “nearer” than changing income, and less fit to suggest. The choice therefore cannot be outsourced to “mathematically nearest” in silence.

Studying it

Enumerate several feasible counterfactuals for the same denial; show only one at random, or show one and mark “there are other roads.” Watch which repair people choose, what they infer about the system’s intent, and whether they feel a life has been assigned. Independent variables: display strategy, distance metric, whether alternatives are disclosed. Dependent variables: type of repair, inferred intent, sense of fairness.

Include a contrast that lays out all feasible roads (when the count is scannable). Whether inferred intent drops after laying-out is this entry’s core gap.

Where it stops holding

Sometimes there is only one feasible counterfactual and the choice does not arise. When the count is too large to scan, laying-out becomes noise and a pick is still required — but the picking rule must be written down. If users can probe for themselves (drag a field and watch the output), some of the power of which to present is handed back and this entry weakens. The risk of gaming is another entry: this one is only that the choice carries value, not that it will be used to slip through.

Applying it

• Do not throw a single “nearest” without saying it was selected. When two or more can be given, give different directions (income / debt) and write “either may flip; which you take is your decision.”
• Put fields excluded from the distance metric (sensitive proxies) into the generation constraint; do not count on users afterwards not to take that road.
• In review, ask why this one was lit and not that one. If there is no answer, the value judgement is in a dark box.
• Check: give different counterfactuals for the same denial and see whether answers to “what the system wants me to do” move with them. If they move, the presentation choice is already living their life for them.

Related

• Same group: L5.08.1 A counterfactual explanation says which input changes would change the outcome · L5.08.2 The explanation is actionable only when the named input is one the user can change · L5.08.4 Counterfactual explanations leak decision rules and may be used to game them · L5.08.5 A counterfactual assumes other conditions held constant, which rarely holds in practice
• Nearby: L5.05 The Moderation Principle of Transparency · L5.02 Local and Global Explanations · L5.01 Types of Explainability
• Search terms: choice among counterfactuals · recourse · nearest counterfactual