Direct manipulation depends on that threshold holding

What it is

Dragging a file into a folder, pulling a slider to a new value, moving an anchor on a canvas — these are called direct manipulation because the object appears to change continuously under the hand. That appearance is pinned to the instantaneous threshold: hold the threshold and the object feels held; miss it and the same gesture collapses into “I issued a move command, then waited for the picture to catch up”. Direct manipulation is not a visual style. It is a latency contract: the object has to follow inside the instantaneous window or the contract is void.

This is not about dragging needing tighter tracking than a click — that is a precision problem. It is about the interaction paradigm itself: if instantaneous binding cannot be kept, direct manipulation does not stand.

Why it happens

Direct manipulation asks that objects stay visible, operations stay reversible, and change stay incremental. Each of those ties feedback to the hand’s clock. While the hand is still moving, the object must already be moving, or the internal model cannot treat screen coordinates as an extension of the limb. Delay cuts the extension: the hand has arrived, the object is still on the previous frame, and the brain switches to prediction — “I have probably moved it there”. Prediction error accumulates, and people change strategy: smaller moves, stepwise clicks, or a numeric field. Manipulation retreats into commands.

When Shneiderman listed the conditions, “rapid, incremental, reversible operations on visible objects” sat together. Drop “rapid” and incrementality and reversibility can remain, yet the object is no longer in the hand. An interface can look like direct manipulation — large icons, draggable layers — and still play as a delayed command line once latency leaves the window.

Studying it

Compare the same task when the object follows instantly versus when the gesture is sampled and the object jumps to the endpoint on release. Classic signs: how many segments the action splits into, whether dragging is abandoned for numeric input, whether people still say “I put it…” rather than “I told it to go…”.

Independent variables: whether object follow lands inside the instantaneous window, whether a single commit happens only on release, whether the object is visible mid-gesture. Dependent variables: duration of continuous drags, rate of switching to indirect controls, ratio of manipulation verbs to command verbs in speech.

Lab drop targets are often large and heavily practised. On a real canvas objects are small and snapping is active; latency can be masked or amplified by snap. Do not declare the paradigm intact from large-target trials alone.

Where it stops holding

Menu picks, dialog confirms and one-shot submits were never direct manipulation; the instantaneous threshold is a bonus, not a condition of existence. Stylus drawing, drag-to-reorder and window resizing are the harshest contracts; ticking a checkbox is far more lenient. Remote desktop and cloud rendering put pixels on the other side of a network, so the contract almost never holds of the real object — only of a local proxy (a ghost, a preview). The user is manipulating the proxy. Pointerless, speech-only interfaces have no limb extension and no such contract.

Applying it

• For any interaction sold as “the object is in the hand” (canvas, timeline, layers, drag-reorder), compute object motion locally. Do not wait for the server to confirm every frame during the gesture.
• The server may validate and roll back on release, but the gesture needs a live local object while it runs. Without one, do not implement a drag; use select-and-confirm.
• If the architecture cannot follow instantly, do not borrow direct-manipulation language (grab cursor, live preview, object glued to the finger). Use an explicit “move to here” command so the contract is not faked.
• How to check: on the target device, drag an object across the screen. A stable spatial gap between finger and object, or people switching to tap-to-place, means direct manipulation does not hold on that device.

Related

• Same group: I1.01.1 Feedback inside a very short delay is perceived as direct causation · I1.01.2 Beyond that threshold, action and result are perceived as two events
• Nearby: I1.04 Input latency and tracking · I4.09 Tempo and interaction rhythm
• Search terms: direct manipulation · latency requirement · object tracking