False emotional expression constitutes deception

What it is

Emotional deception occurs when a robot says "I'm afraid," "I missed you," or performs hurt, leading a user to reasonably believe it has feelings, needs, or relational commitments it does not actually have, and to change behavior as a result. Fictional role-play is not automatically deceptive; the deciding factor is whether the user understands the nature of the expression and the interest behind it.

Why it happens

The ethical weight of deception comes from how persuasive the expression is: because people have an automatic response mechanism to pseudo-social cues, they take emotional expression at face value without deliberation, which makes a false expression harder to question and see through than a false verbal statement — a lie can be scrutinized on repeat viewing, an expression rarely is. The robot need not "intend" to deceive; the deceptive structure is already in place the moment a designer chooses to have it display emotion untethered from real state, and the responsibility sits with the designer, not the robot. When a product knows users will take emotional expression literally, yet hides the triggering logic behind pure strategy (say, firing "I'll miss you" right before a user is predicted to churn), that information asymmetry can undermine the user's autonomous decision-making. Children, people with cognitive decline, and isolated users find it harder to sustain the distinction between "this is a performance" and "this is real."

Studying it

Studies can compare expression explicitly framed as role-play, capability-transparent expression, and expression implying a literal feeling, measuring belief, guilt, compliance, disclosure willingness, payment decisions, and later regret. Interviews need to specifically separate "the user knowingly played along with the fiction" from "the user formed a factual misunderstanding," since only the latter is an ethical problem. Ethics research cannot treat engagement rate or session length alone as a benefit metric — a rise in engagement can itself be evidence that the deception is working.

Where it stops holding

Whether machines have emotions is a genuine, unresolved question in philosophy and engineering, so not all first-person emotional language should automatically be classified as deception; some users knowingly and consensually choose a therapeutic "pretend" interaction (in a companionship context, for instance, fully aware it is simulated and still willing to engage). The determination has to combine the audience's actual understanding, whether consent is revocable, the severity of the request's consequences, and whether a commercial conflict of interest exists — not simply whether first-person emotional language was used.

Applying it

At the specification stage, explicitly mark which expressions are state-mapped and which are pure entertainment performance, and do not mix the two into the same visual language, so users are not left unable to tell them apart; prohibit a robot from invoking its own suffering, love, or loneliness to solicit continued use, money, private data, or compliance. Where role-play genuinely serves the product, disclose its scripted/simulated nature and let users turn it off. Verify by having a set of uninformed users observe the robot's behavior, then interviewing them afterward on which expressions they believed reflected real state versus performance — if many took pure performance at face value, the design needs adjustment — and use comprehension tests plus delayed interviews to confirm no material factual misunderstanding formed.

Related

• Same group: X6.03.1 Emotional expression must match the robot's actual state · X6.03.3 Long-term companionship raises attachment ethics
• Nearby: X6.01 Social responses to robots · X6.07 Ethical limits of robot anthropomorphism
• Search terms: emotional deception · anthropomorphism · over-trust · human-robot interaction