5 ms·
To be clear, by "learned to read" – it actually learned to OCR text, unsupervised. That is pretty impressive, IMO. As for whether NNs are doing symbolic manipu
by moyix 6y ago
To be clear, by "learned to read" – it actually learned to OCR text, unsupervised. That is pretty impressive, IMO.
As for whether NNs are doing symbolic manipulation – I'm not so sure that they aren't. DALL-E, the sibling of CLIP, can do some things that IMO are hard to explain if there's nothing symbolic going on, like the examples in this post [1]. Of course we can't say it's doing what humans do (we don't know what humans do!) but it looks, to me, like more than pure pattern matching. nostalgebraist had some good thoughts on this (and actually so too did Gary Marcus well before him) [2].
[1] https://openai.com/blog/dall-e/ https://openai.com/blog/dall-e/
[2] https://nostalgebraist.tumblr.com/post/189965935059/human-psycholinguists-a-critical-appraisal https://nostalgebraist.tumblr.com/post/189965935059/human-ps...
- a1369209993 6y ago> by "learned to read" - it actually learned to OCR text, unsupervised. I think their claim is that it did not learn to OCR text, but merely to recognise certain specific images that happen to be things a human would interpret as specific bits of text.
- the8472 6y agoFrom the CLIP article: > While CLIP’s zero-shot OCR performance is mixed, its semantic OCR representation is quite useful. When evaluated on the SST-2 NLP dataset rendered as images, a linear classifer on CLIP’s representation matches a CBoW model with direct access to the text. CLIP matches other models in text sentiment analysis when presented with image renderings of the text while the other model had direct access to the text. It also does MNIST. It might not be the best OCR software ever, but that's besides the point. It's huge advancement in generality on visual tasks and part of that generalization is some reading capability.
- moyix 6y agoYes, to be extra clear, if that is their claim, their claim is wrong.