P4.07.1Designer responsibility for foreseeable harmdesign

Executors bear responsibility for foreseeable harm

Aliases: foreseeable harm · professional responsibility · moral responsibility in engineering

What it is

"I was only implementing" transfers no responsibility: when the foreseeable consequences of a design or implementation include harm to users, the engineers and designers who touched it bear responsibility proportionate to what they knew and what they could change. Responsibility for foreseeable harm turns on two dimensions — your information at the time and your control — with more knowledge and more control meaning more responsibility. Professional codes of ethics broadly accept this structure: practitioners' obligations run to public wellbeing ahead of the employer.

Why it happens

Organizations systematically dilute individual responsibility: it gets shredded along the division of labor (proposer, reviewer, implementer, releaser each hold a segment), and each segment alone "wasn't the final call"; harm is stretched over time (months between launch and incident), and causal feeling decays with distance; and role-based moral buffers ("the algorithm decided," "the data said," "the metric said") give executors impersonal attribution outlets. What counters all this is foreseeability as an anchor: once harm can be concretely described — which choice at which step damages which users, how severely, how reversibly — the defense of "couldn't have foreseen" collapses, leaving only "didn't look." Automated systems add another layer worth flagging: the more autonomous a system appears, the more readily human responsibility is reassigned post-hoc to "the technology," though behind every autonomous behavior stand traceable design choices; in accident analyses the operator often becomes the "moral crumple zone" absorbing all blame despite having no control over the system's design.

Where it stops holding

Responsibility has a ceiling: executors are not accountable for unforeseeable harm or for information deliberately withheld from them; responsibility's weight is reasonably measured by the product of knowledge and authority. Attribution is also distributed — emphasizing individual responsibility does not unload organizational and managerial accountability onto the front line; on the contrary, individual responsibility's ethical value lies in supplying evidence for organizational accountability: allies, not substitutes. Legal and ethical obligations also differ: legal immunity is not ethical innocence, and ethics usually demands more than compliance.

Applying it

  • Make "foreseeable harm" a fixed question in design reviews: at worst, whom does this choice hurt, how badly, how reversibly; being unable to name a harm is not absence of harm but absence of thought.
  • Clarify your own position of authority in the process: what you know, what you can change, what you can block; responsibility updates with authority and is recorded.
  • Convert anticipated risks from hallway talk to writing (risk registers, review records), so "I raised it" is verifiable rather than post-hoc dispute.
  • Verify: at the next incident retrospective, walk each link of the division of labor — which link held both information and authority yet did not act; use the conclusion to revise that link's responsibility conventions, not merely to blame an individual.

Related

  • Same group: P4.07.2 Metric pressure is not a defense · P4.07.3 Refusing and recording dissent are viable professional acts
  • Adjacent: P4.14.4 Retrospection depends on a written decision trail · P4.01 Value-sensitive design
  • Search terms: foreseeable harm · professional responsibility · moral crumple zone

Cards in the same group

Quick Actions

Share

Share this page

ios_share

https://hci.top/en/handbook/P4.07.1