Failed fusion should fall back to one modality, not guess

What it is

When fusion fails, fall back to the literal interpretation of one modality rather than guessing intent. If the system picks an object from an uncertain reference and executes immediately, users often never notice the wrong choice. An explicit fallback preserves user control at the moment of failure.

Why it happens

Guessing is dangerous because its errors are hard to see: the action ran, the interface looks normal, and the user may only notice on later inspection. Falling back exposes the uncertainty—executing only what the speech made explicit, or asking the user to confirm the object. The cost is extra steps, but it is paid before the error rather than after. For irreversible actions, fallback is close to the only sound choice, since post-hoc recovery may not be possible.

Studying it

Compare guessing and fallback: under uncertain reference, measure misoperation rate, the proportion users notice, and completion time. Variables include action reversibility, candidate count, and degree of uncertainty. Include the proportion of unnoticed errors, which is the real cost of guessing.

Where it stops holding

When actions are reversible and cheap, the extra steps of fallback may not be worth it, and executing a guess with a prominent undo is better. When the candidate is unique or uncertainty is negligible, fallback is unnecessary. For critical actions users generally expect confirmation over automatic execution, so fallback gains value as consequences grow.

Applying it

• Require confirmation for irreversible actions when reference is uncertain instead of choosing an object automatically.
• For reversible actions, execute the guess but provide a prominent undo.
• When falling back, state what the system could not determine so users know what to supply.
• Verification: under uncertain reference, measure the proportion of unnoticed errors; if guessing leaves it high, switch to confirmation or fallback.

Related

• Within the group: D5.02.1 Speech supplies the action, pointing supplies the object · D5.02.2 Fusion requires alignment within a time window
• Adjacent: D5.08.1 When fusion fails, fall back to the literal reading of one modality · D4.07.4 Adjudication should let users perceive that a conflict occurred
• Search terms: silent guessing · explicit fallback · confirmation