3 ms·
> by "learned to read" - it actually learned to OCR text, unsupervised. I think their claim is that it did not learn to OCR text, but merely to recognise certa
by a1369209993 6y ago
> by "learned to read" - it actually learned to OCR text, unsupervised.
I think their claim is that it did not learn to OCR text, but merely to recognise certain specific images that happen to be things a human would interpret as specific bits of text.
- the8472 6y agoFrom the CLIP article: > While CLIP’s zero-shot OCR performance is mixed, its semantic OCR representation is quite useful. When evaluated on the SST-2 NLP dataset rendered as images, a linear classifer on CLIP’s representation matches a CBoW model with direct access to the text. CLIP matches other models in text sentiment analysis when presented with image renderings of the text while the other model had direct access to the text. It also does MNIST. It might not be the best OCR software ever, but that's besides the point. It's huge advancement in generality on visual tasks and part of that generalization is some reading capability.
- moyix 6y agoYes, to be extra clear, if that is their claim, their claim is wrong.