Character segmentation errors cascade and spoil recognition of everything after

What it is

Most recognition pipelines first cut a line of ink into character hypotheses, then classify. One wrong cut—failing to break, or breaking where it should not—shifts every later window: one character is read as two, or two are glued as one, and characters after that are recognised against the wrong boundaries. Errors walk like dominos. That is a structural failure inside recognition, not how people later edit the text.

Why it happens

Segmentation leans on gaps, lifts, projection histograms, or learned boundaries. Joined writing, uneven spacing, and punctuation stuck to a glyph kill the gap signal. Over-segmentation splits a character; the classifier sees broken parts and emits two confident wrong glyphs. Under-segmentation feeds two characters to one classifier and emits a chimera. If later decoding assumes “the last boundary was right”, it never slides the window back, and the line collapses from that cut onward. A language model can pull back at word level, but if segmentation also wrecked word boundaries, the model sees tokens of the wrong length. Search-based recognition (keeping several segmentation hypotheses and scoring them) can stop the cascade, at a cost in compute and latency. Vertical Chinese, English cursive, and digit strings have different cues; one gap threshold cannot be shared.

Studying it

On a gold segmentation, inject over- and under-cuts and watch later CER climb to the right of the first error. Compare greedy cuts with beam search over hypotheses.

Independent variables: segmentation strategy, spacing, how joined the writing is, language. Dependent measures: segmentation F1, subsequent CER from the first bad cut, beam width versus latency.

Overall line CER mixes segmentation disasters with single-glyph confusions; align curves on the first-cut position.

Where it stops holding

Sequence models that recognise a whole line without a hard character cut have a different cascade (alignment drift), but a long line can still die from one alignment error. A single isolated-character field has no segmentation problem. Writing in boxes (a grid) hands segmentation to layout; failures become “wrote outside the box” rather than a cascade.

Applying it

• Use multi-hypothesis segmentation on free writing; do not lock the first greedy cut.
• Let people merge and split segmentation boxes on the ink—a structural edit of the recognizer, not ordinary typing.
• Be even more conservative on digits and password-like fields with no lexicon; slower is better than a shifted string.
• How to check: join two characters mid-line on purpose and see whether the rest of the line becomes garbage. With a beam, collapse should stop near the join. With merge/split, people should fix that one cut without rewriting the line.

Related

• Same group: C5.14.1 Online uses stroke order and dynamics; offline uses the image · C5.14.3 Recognition latency trades against accuracy · C5.14.4 Cursive is much less accurate than regular script
• Adjacent: C5.05 Handwriting Recognition and Correction · C5.15 Handwriting Correction and Post-editing
• Search: character segmentation · error cascade · over-segmentation