trocr

Definition

An OCR model from Microsoft built entirely out of Transformers: a pre-trained image Transformer encodes the cropped text image and a pre-trained text Transformer decodes the characters. Dropping the usual CNN-plus-CTC recogniser lets the decoder bring language modelling to bear on ambiguous glyphs. Variants cover printed, handwritten, and scene text. It is still a recogniser fed line crops, not an OCR-free document reader.