Complex edits are less efficient to express in speech than by direct manipulation

What it is

Replacing one word can be a spoken command. Swapping two blocks, changing heading level, filling three columns of a table, or fixing indentation in code—these complex edits—take more steps and more failure surface in speech than in direct manipulation (a touch drag, a shortcut, a mouse marquee). The gap is the cost of saying a spatial structure as a linear instruction. More accurate recognition will not erase it.

Why it happens

Direct manipulation holds objects in visual space: drag a span, marquee, multi-caret. Speech must serialise the same structure into talk, and the parser must rebuild it. The more two-dimensional the structure (table, tree, layout), the longer the serialisation, and every step can be broken by recognition or scope error. Undo, redo, and find-replace are chords on a keyboard and a multi-turn dialogue in speech. Working memory holds “what command do I say next,” not “where does the hand go.” Complex edits should therefore change channel rather than give every editor menu an utterance. This sits beside the missing precise caret: that is a missing fine grain; this is throughput of the operation set on the channel.

Studying it

Pick structured tasks (swap paragraphs, change list level, fill a table, a light refactor) and compare voice-only, keyboard-only, and voice plus touch on time and errors. Count how many steps are spent describing location rather than changing content. A single typo is not a stand-in for complex editing.

Where it stops holding

For people who cannot use their hands, slower speech is still the primary channel; an efficiency comparison is not a reason to withhold it, but a reason to offer better structural navigation. A well-practised oral macro (“turn these three paragraphs into bullets”) can crush one class of complex operation into a sentence: specialised grammar, not general talk. One-off exploratory layout is still faster by direct manipulation.

Applying it

• Cover with voice only high-frequency, chunk-sized operations whose objects can be named; leave fine table, tree, and code structure to keyboard or pointer.
• When several turns in a row are spent describing location, prompt a switch to direct manipulation and keep the current selection.
• Accept on real documents: if the voice path for complex edits is clearly longer than touch, do not market “you can edit everything by mouth.”

Related

• Same group: C7.13.1 Spoken correction depends on saying a replacement command rather than editing text directly · C7.13.2 The system must decide whether a correction targets recent text or an arbitrary earlier span · C7.13.3 Voice editing lacks the precise caret placement of keyboard editing
• Adjacent: C7.17 Switching Between Voice and Keyboard · C7.14 Command Grammar versus Free Expression
• Search: voice editing throughput · direct manipulation · structured edit