Transparency does not necessarily produce trust; exposing complexity can lower it

What it is

Radiology unfolds the ensemble’s votes: three backbones disagree, the threshold was tuned by hand. After seeing that, doctors are less willing to hand over than when they saw one clean score. Transparency does not automatically produce trust; lighting up complexity sometimes presses trust down.

Trust tracks “dare I still hand this over,” not “how much inside did I see.” Seeing can become “this is messier than I thought.”

Why it happens

People use simplicity as a cue to competence. A clean score looks like a tool held steadily; a pile of submodels that fight looks unfinished. Complexity is a fact, and the fact can lower handover — which is not the same direction as “hiding failures wrecks calibration.” There, failure samples are missing; here, structure people cannot digest is being read as immaturity.

Lee and See’s calibration wants a match to reliability, not a match to the parts list. If the parts list cannot be rewritten as a range of reliability, it is only showing that the engineering has not converged. Showing non-convergence is read as “not ready,” even when the actual hit rate is not bad.

Studying it

Same reliability: one group sees a calibrated performance summary, another sees internal parts (ensemble votes, hyperparameters, contradictory sub-scores). Measure handover and estimates. Independent variables: how much complexity is exposed, whether a performance summary is also given, whether the audience are domain experts. Dependent variables: handover, estimate of reliability, rate of reading complexity as “immature.”

Experts can sometimes read parts as “so there is a backup,” and handover rises rather than falls. Report by audience.

Where it stops holding

When complexity is exactly what explains a capability boundary (“when image quality is poor the three will scatter; treat as undecided”), exposing it can support calibration and this entry flips. If public communication is already handling a scandal, hiding parts will hurt more. This entry does not argue for hiding failures to protect trust; it argues that a parts list is not an input to trust. A success streak pushing trust high, or one severe miss emptying it, are different processes.

Applying it

• Default to performance and scope, not parts. Parts go to people who must fix the system or audit it.
• If internals must be exposed, first translate them into a meaning for handover (“when they disagree, do not treat this as a firm diagnosis”), rather than pasting the raw vote table.
• Do not treat “we are transparent” as a trust metric. The trust metric remains the gap between handover and reliability.
• Check: two versions, a clean summary versus unfolded ensemble detail; ask whether they still dare hand over the next case. Handover down and hit rate unchanged means transparency, on this entry, has hurt trust.

Related

• Same group: L5.05.1 Excess technical detail does not produce understanding · L5.05.2 Transparency must serve the user's next decision · L5.05.3 Information that cannot be acted on is noise · L5.05.4 Transparency is sufficient only if the user can change the next action from it · L5.05.5 Excess detail crowds out attention needed for key information · L5.05.6 Transparency and usability are in tension; full disclosure makes the interface unusable · L5.05.8 Different roles should receive different depths of explanation, not the same text
• Nearby: L5.03 Trust Calibration · L5.09 Overtrust and Trust Collapse · L5.02 Local and Global Explanations
• Search terms: transparency without trust · complexity and trust · algorithm aversion