Recognition latency and accuracy trade off; live recognition usually gives up some accuracy

What it is

Emitting characters while writing requires a result before the stroke, and the context, are finished. Waiting for a whole line can use a larger model, a sentence language model, and several segmentation searches, so accuracy is usually higher, but no character appears. A live path cuts search, uses a smaller model, and sees little right context, so accuracy drops. That is the recognizer’s own time–quality trade, not the tens of milliseconds of ink following.

Why it happens

Latency is waiting for more ink (word boundary, timeout) plus compute (forward pass, beam width). Live tactics include: a shallow model on every stroke or character; left-side recognised text as context and empty on the right; a capped beam. Shallow models are weaker on cursive and look-alikes. Without right context, the first Latin letters or a Chinese left radical are committed too soon, and the rest of the glyph then wants a change. A final full-line “cool-down” model can raise committed accuracy, but the screen has already flashed an error, and people may have edited or been misled. A short window after lift is a compromise: more accurate than per-stroke, faster than a whole line. Latency hits the writing loop when characters arrive so late that people do not wait, keep writing, and the result lands far from fixation.

Studying it

Sweep beam width, trigger grain (per stroke / character / line), and model size on one engine, and plot CER against latency. Log time to first visible character and committed CER after cool-down.

Independent variables: trigger grain, beam, full-line cool-down, device compute. Dependent measures: first-character delay, live CER, committed CER, whether people fix errors before cool-down.

The curve must be measured on the target hardware. Server accuracy does not transfer to on-device live recognition.

Where it stops holding

Dictation-style “write a passage, then emit” can take the high-accuracy path and should not be yoked to the live curve. Imported scans have no live pressure. With abundant compute the trade weakens, but waiting for right context remains an information trade. Early characters that cannot be edited turn the trade into irreversible error.

Applying it

• Decode live with a shallow model; cool down once on the full line. Cool-down changes must be visible and must not interrupt the next stroke.
• Split “time to a character” and “final CER” as two metrics; do not let one excuse the other.
• Short field input can wait for lift; it need not be per-stroke.
• How to check: plot the CER–latency curve and pick the product point. Turn cool-down off and see how much live CER worsens. Lengthen waiting and see whether people have already written elsewhere. Measure on-device and server curves separately; do not mix them.

Related

• Same group: C5.14.1 Online uses stroke order and dynamics; offline uses the image · C5.14.2 Segmentation errors cascade · C5.14.4 Cursive is much less accurate than regular script
• Adjacent: C5.08 Inking Latency · C5.05 Handwriting Recognition and Correction
• Search: recognition latency · accuracy tradeoff · incremental decode