OCR fine-tuning must preserve label tokens and be gated end-to-end

General lesson: fine-tuning an OCR recognizer only on identifier digits can improve crop-level numeric accuracy while catastrophically degrading the label tokens and full-line behavior that an end-to-end extractor depends on. Include the surrounding label vocabulary (for example DOT/MC), select checkpoints on a disjoint development set, and always run a small end-to-end smoke test before spending the sealed validation set. Even after mixing label crops back in, judge the model by paired end-to-end exact match, wrong-accepted-value rate, and digit deletion/leading-zero regressions—not crop accuracy alone.

Applicability and limits: this applies to weakly supervised OCR adaptation where detection, recognition, and downstream label-to-value association are separate stages. It does not show that all recognition-only tuning fails; the result depends on crop construction, label quality, augmentation, freezing strategy, and the base model.

Applies when: adapting an OCR recognizer to structured identifiers whose downstream parser relies on nearby labels. Keywords: OCR fine tuning catastrophic forgetting labels digits leading zeros validation