3 ms·
From the CLIP article: > While CLIP’s zero-shot OCR performance is mixed, its semantic OCR representation is quite useful. When evaluated on the SST-2 NLP data
by the8472 6y ago
From the CLIP article:
> While CLIP’s zero-shot OCR performance is mixed, its semantic OCR representation is quite useful. When evaluated on the SST-2 NLP dataset rendered as images, a linear classifer on CLIP’s representation matches a CBoW model with direct access to the text.
CLIP matches other models in text sentiment analysis when presented with image renderings of the text while the other model had direct access to the text. It also does MNIST.
It might not be the best OCR software ever, but that's besides the point. It's huge advancement in generality on visual tasks and part of that generalization is some reading capability.