Window length should be tuned per modality pairing, not fixed universally

What it is

Fusion windows should be calibrated for each modality pairing, task, and device chain rather than set to one global millisecond value. Speech plus pointing, gaze plus selection, and haptic cue plus verbal confirmation differ in signal shape and latency source. A universal value is too narrow for slow channels and too wide for fast ones.

Why it happens

Each pairing combines three time sources: human expression, sensor and recognizer processing, and delivery into the fuser. Speech contains words, pauses, and confidence updates; gestures run from movement onset to completed deixis; gaze has sampling noise; local buttons are fast and clearly bounded. Treating all as one arrival time excludes slow channels while leaving fast channels exposed to interference. Task context matters too: continuous revision, dragging, and accessible alternative input may need longer chains than one-shot commands.

Studying it

Build a timing distribution per pairing: collect onset, completion, recognition confidence, and outcome logs, then estimate natural intervals and device delay at the 50th, 90th, and 99th percentiles. Perturb latency around those points in controlled tests and observe where binding accuracy and false fusion cross. Variables include utterance length, motor ability, network or local inference delay, concurrent tasks, and assistive technology. Report missed fusion and false fusion together; do not optimize overall success alone.

Where it stops holding

Per-pairing tuning is not license for endless adaptation. If the window drifts on every input, users cannot form a stable expectation and failures become hard to reproduce. Older adults, people with motor disabilities, and assistive devices may produce longer intervals and should not be treated as outliers; critical operations still cannot bypass confirmation because the system is waiting. With uncontrollable network delay, visible waiting and cancellation are safer than silently binding uncertain candidates.

Applying it

• Maintain separate window configurations per supported pairing, with rationale, applicable task, and version.
• Include recognition, queue, and user-interval percentiles in the budget, with conservative caps for high-consequence tasks.
• Make waiting, timeout, and cancellation visible so long windows do not become invisible state.
• Verification: test slow, fast, and assistive-input users; compare missed fusion, false fusion, and completion time, then return to the affected pairing when either error rises.

Related

• Within the group: D5.07.1 Multimodal input must fall within one temporal window to count as one intention · D5.07.2 Too narrow a window misreads normal sequential expression as separate commands
• Adjacent: D4.07.1 High-risk operations need an explicit feedback time budget · D5.10.3 Detection latency changes perceived responsiveness
• Search terms: modality-specific timing · latency budget · fusion tuning