Providing explanations does not necessarily weaken the bias

What it is

Putting a reason, a feature-importance bar, or a “because…” next to advice does not mean people will check. An explanation can be read as decoration on the advice, used to accept faster, not to challenge. Explanation does not necessarily reduce automation bias splits “explainable” from “will review”: the first is a shape of system output; the second is whether the person did an independent check.

A fluent explanation often makes bad advice look better. The door for bias gets wider, not narrower.

Why it happens

Checking asks “is the world like this”; explanation asks “why does the system say so.” They use different material. If the explanation is the model’s self-report, it is homologous with the advice and cannot serve as independent evidence. People still read “there is a reason” as “already verified.” When the UI parks the explanation directly under the advice, in declarative sentences, in the brand voice, the reading order becomes advice → reason → release; the source never gets a slot.

Explanation can also spend the time that should have gone to the raw object. In a short window, an extra paragraph equals one fewer check.

Studying it

Hold the same correct and incorrect advice, and compare no explanation, a true explanation, and a pseudo-explanation that agrees with the advice but conflicts with the world. Dependent variables: follow-through on the error, whether the source is opened, decision time, subjective “I understand so I can pass.” Independent variables: whether the explanation is homologous with the advice, whether it is causal or a correlation ranking, length.

Separate “read the copy” from “checked the world.” If people say “the explanation made me comfortable” and checking drops at the same time, the explanation is helping the bias, not dismantling it.

Where it stops holding

When the explanation points at an input the person can change immediately (a counterfactual, an actionable condition) and they actually change it and look again, explanation can interrupt following. That is already close to a trial, not a sidebar. Advice with no explanation at least does not pretend to have been verified. This entry does not treat whether the bias intensifies as accuracy rises; it treats only the act of adding copy.

Applying it

• Do not treat explanation as a stand-in for checking. The release condition remains contact with the raw object; explanation is an optional note at most.
• When explanation and advice share a source, label it as “what the system says,” not as a list of evidence.
• In a short window, make the source visible first, not the reason fully read. A reason can fold if unread; the source cannot fold until after release.
• Check: give advice with a polished reason that conflicts with the source. If the release rate is higher than the same conflict with no reason, the explanation is reinforcing the bias. Remove the reason and measure checking again — if checking rises, the explanation had been spending the checking time.

Related

• Same group: L4.02.1 People tend to accept system advice without checking · L4.02.2 The bias strengthens as the system gets more accurate
• Nearby: L5.01 Types of Explainability · L5.03 Trust Calibration · L5.08 Counterfactual Explanations
• Search terms: automation bias · explainability and compliance · automation misuse