7 ms·

I think people are focusing too much on the attacks. Imo those are a sideshow. It's even questionable whether they can be called attacks when you're asking it t

I think people are focusing too much on the attacks. Imo those are a sideshow. It's even questionable whether they can be called attacks when you're asking it to recognize an image and it recognizes something that's on the image, just not in the mode you expected or the object you assumed was of interest. It's way more important that CLIP learned to read (crudely) as part of image recognition, like GPT3 learned addition as part of NLP. It has neurons for abstract concepts. It does amazingly on a bunch of other tasks without being explicitly trained for them. It integrates multiple senses. And it already does very simple image descriptions[0]. In a few scaling iterations it might have a better understanding of a scene and describe it as "an apple on a wooden table with a piece of paper reading 'ipod' attached", end of confusion. TL;DR, they're gaining new capabilities, it's unsurprising that they have not mastered them to superhuman levels yet. [0] https://openai.com/blog/clip/#zero-shot-probabilities https://openai.com/blog/clip/#zero-shot-probabilities

beaconstudios 6y ago
> It's way more important that CLIP learned to read (crudely) It learned to detect the features that make up the word "iPod", and match them against other features. I don't see that as being any different than detecting the features that make an apple, or a car or a face. It's just detecting the minimum number of features to distinguish the images in a training set. I'm generally disappointed with how breathlessly this forum gets excited about NN software. It doesn't 'get' images as a whole and compare them. There's no symbolics going on underneath, it's just picking up 'something' in an image and matching it with 'something else'. It's clearly nothing like how we see, let alone how we think or imagine. It's impressive and a useful innovation for sure, but it's not even on the ladder to our capabilities yet and I'm not convinced that it ever will be.
- moyix 6y ago
  To be clear, by "learned to read" – it actually learned to OCR text, unsupervised. That is pretty impressive, IMO. As for whether NNs are doing symbolic manipulation – I'm not so sure that they aren't. DALL-E, the sibling of CLIP, can do some things that IMO are hard to explain if there's nothing symbolic going on, like the examples in this post [1]. Of course we can't say it's doing what humans do (we don't know what humans do!) but it looks, to me, like more than pure pattern matching. nostalgebraist had some good thoughts on this (and actually so too did Gary Marcus well before him) [2]. [1] https://openai.com/blog/dall-e/ https://openai.com/blog/dall-e/ [2] https://nostalgebraist.tumblr.com/post/189965935059/human-psycholinguists-a-critical-appraisal https://nostalgebraist.tumblr.com/post/189965935059/human-ps...
  - a1369209993 6y ago
    > by "learned to read" - it actually learned to OCR text, unsupervised. I think their claim is that it did not learn to OCR text, but merely to recognise certain specific images that happen to be things a human would interpret as specific bits of text.
    - the8472 6y ago
      From the CLIP article: > While CLIP’s zero-shot OCR performance is mixed, its semantic OCR representation is quite useful. When evaluated on the SST-2 NLP dataset rendered as images, a linear classifer on CLIP’s representation matches a CBoW model with direct access to the text. CLIP matches other models in text sentiment analysis when presented with image renderings of the text while the other model had direct access to the text. It also does MNIST. It might not be the best OCR software ever, but that's besides the point. It's huge advancement in generality on visual tasks and part of that generalization is some reading capability.
    - moyix 6y ago
      Yes, to be extra clear, if that is their claim, their claim is wrong.
gwern 6y ago
> I think people are focusing too much on the attacks. Imo those are a sideshow. It's even questionable whether they can be called attacks when you're asking it to recognize an image and it recognizes something that's on the image, just not in the mode you expected or the object you assumed was of interest. Yeah, as expected, https://twitter.com/NoaNabeshima/status/1368662246885265409 https://twitter.com/NoaNabeshima/status/1368662246885265409 https://youtu.be/Rk3MBx20z24?t=35 https://youtu.be/Rk3MBx20z24?t=35 it goes away with a more sensible set of inputs. CLIP just turns out to be more like GPT-3 in that you don't input 'classes', you do 'prompt programming'. Looking at the relative activation of 'ipod' is just wrong; you are asking a stupid question and acting surprised when you get a reasonable answer which happened to not telepathically read your mind.

beaconstudios 6y ago

> It's way more important that CLIP learned to read (crudely) It learned to detect the features that make up the word "iPod", and match them against other features. I don't see that as being any different than detecting the features that make an apple, or a car or a face. It's just detecting the minimum number of features to distinguish the images in a training set. I'm generally disappointed with how breathlessly this forum gets excited about NN software. It doesn't 'get' images as a whole and compare them. There's no symbolics going on underneath, it's just picking up 'something' in an image and matching it with 'something else'. It's clearly nothing like how we see, let alone how we think or imagine. It's impressive and a useful innovation for sure, but it's not even on the ladder to our capabilities yet and I'm not convinced that it ever will be.

moyix 6y ago
To be clear, by "learned to read" – it actually learned to OCR text, unsupervised. That is pretty impressive, IMO. As for whether NNs are doing symbolic manipulation – I'm not so sure that they aren't. DALL-E, the sibling of CLIP, can do some things that IMO are hard to explain if there's nothing symbolic going on, like the examples in this post [1]. Of course we can't say it's doing what humans do (we don't know what humans do!) but it looks, to me, like more than pure pattern matching. nostalgebraist had some good thoughts on this (and actually so too did Gary Marcus well before him) [2]. [1] https://openai.com/blog/dall-e/ https://openai.com/blog/dall-e/ [2] https://nostalgebraist.tumblr.com/post/189965935059/human-psycholinguists-a-critical-appraisal https://nostalgebraist.tumblr.com/post/189965935059/human-ps...
- a1369209993 6y ago
  > by "learned to read" - it actually learned to OCR text, unsupervised. I think their claim is that it did not learn to OCR text, but merely to recognise certain specific images that happen to be things a human would interpret as specific bits of text.
  - the8472 6y ago
    From the CLIP article: > While CLIP’s zero-shot OCR performance is mixed, its semantic OCR representation is quite useful. When evaluated on the SST-2 NLP dataset rendered as images, a linear classifer on CLIP’s representation matches a CBoW model with direct access to the text. CLIP matches other models in text sentiment analysis when presented with image renderings of the text while the other model had direct access to the text. It also does MNIST. It might not be the best OCR software ever, but that's besides the point. It's huge advancement in generality on visual tasks and part of that generalization is some reading capability.
  - moyix 6y ago
    Yes, to be extra clear, if that is their claim, their claim is wrong.

gwern 6y ago

> I think people are focusing too much on the attacks. Imo those are a sideshow. It's even questionable whether they can be called attacks when you're asking it to recognize an image and it recognizes something that's on the image, just not in the mode you expected or the object you assumed was of interest. Yeah, as expected, https://twitter.com/NoaNabeshima/status/1368662246885265409 https://twitter.com/NoaNabeshima/status/1368662246885265409 https://youtu.be/Rk3MBx20z24?t=35 https://youtu.be/Rk3MBx20z24?t=35 it goes away with a more sensible set of inputs. CLIP just turns out to be more like GPT-3 in that you don't input 'classes', you do 'prompt programming'. Looking at the relative activation of 'ipod' is just wrong; you are asking a stupid question and acting surprised when you get a reasonable answer which happened to not telepathically read your mind.