3 ms·

That can not be the important aspect, at least not in the general case. Show me only a few good photos of an object I have never seen - and one may be sufficien

That can not be the important aspect, at least not in the general case. Show me only a few good photos of an object I have never seen - and one may be sufficient if the object is different enough from all the things I know - and it will become almost impossible for you to trick me into confusing it with anything else and certainly not be changing a few pixels in validation images. I may make mistakes if the image quality is bad or the lighting or if you show me an object that is at least visually very similar and where I am unaware of the distinguishing features. No where here are other senses than vision involved.

ianai 9y ago
Yes you have a concept of an object based on all your memories across all senses. It takes all of your senses and their multiple sources to form concepts.
- danbruc 9y ago
  This is true insofar that the huge amount of background knowledge I have and use to build a model of what I see in an image was almost inevitably build over my lifetime with all my senses, but this does not change the fact that I performed the task at hand only visually, i.e. without fusing different senses. The AI is lacking my background knowledge, not senses other than vision. One might argue that building human-level background knowledge requires more than just vision but it seems at least not totally obvious that this is indeed the case.
- TFortunato 9y ago
  This is a very interesting point that I think has more going for it than the poster you are replying to may realize. Related, and apologies I don't have a link, but there have been a few stories circulated of adults who were blind at birth / a young age who had their sight restored at adulthood - They often had great difficulty, or even found it impossible to correlate what they saw with their eyes with the concept of objects that they knew by touch and their other senses.
  - danbruc 9y ago
    I have no link at hand either, but I also read about blind people being unable to transfer their concepts of objects build from touching them to seeing them when their sight was restored. They know what a sharp edge or a pointy thing feels like but they don't know what it looks like. But I think that actually strengthens my point, blind people are capable of learning how different object feel and they recognize them this way but they so just fine without vision and if they suddenly gain access to vision it does not help them. In general having more senses will help to more reliably identify objects because you have access to more features to differentiate them, for example distinguishing materials from imitations just by vision can be hard or maybe even impossible. But when you can also touch them you gain a lot of new information about surface structure, hardness, thermal conductivity and so on and you can easily distinguish between, for example, real stone and plastic or wood with a stone print on it. But just because it is advantageous in the general case to have access to more than one sense that does not imply that it is necessary for a specific task.
    - TFortunato 9y ago
      Apologies ahead of time. I'm typing this on a small screen phone, and trying to not ramble :-) We might be more on the same page than I realized On the one hand, I agree that it doesn't imply that it is necessary to have access to more than one sense. On the other hand I do think that having access to more senses may make the problem a LOT easier, especially during the training process for teaching systems to do things like try to distinguish objects by sight. Don't get me wrong, the progress made so far has been incredible, but it isn't surprising that we are eventually running into limits given the limited amount of data these systems have to train on. Sort of riffing off of this - In particular, my sense is that when you are training on 2-D pixel data only, it is no wonder that no matter how clever your network is to extract high-level features from the data, you are going to run into these kind of issues. We are asking the system to perform a task (describing, classifying, etc.) about a fundamentally 3-d world, using only a 2-d image, when it has never had a "concept" of the world in 3 dimensions. I think that we take for granted that when we are learning, not only do we get information by seeing, but also touching and manipulating objects and existing in the world around us. We can see things in different lighting conditions, and we can also move our heads and our bodies and around the world and see things from new angles and manipulate them to learn the rich set of correlations between what we see, and what we experience...basically the rich structure of the world around us. A system which is trained on images alone doesn't have the integrated knowledge, which is what I was hoping to get at with the anecdote of adults being cured of blindness. When you look at some of the work we do to augment visual data sets, by scaling images, cropping them, rotating them, skewing them, etc. its basically a poor mimcry of something that humans and animals get just by being in the world...this idea of learning that things remain the same and have a certain structure, regardless of the viewpoint we see them from. (From what I understand, this idea is also part of what inspires capsule networks). Anyways, to come to my main point. Yes, you can absolutely identify objects, even a completely novel object after only seeing it once or twice. But I would argue that this is only because you already have a rich framework of the world as 3-dimensional spatially, along with all of the other priperties that object have that you have learned. So when you see a 2-dimensional picture, your brain can form a 3-d image of the scene, identify materials of objects, etc., which you can leverage. Things that our current AI systems have no idea of... so perhaps its a miracle they made it this far, and no wonder that we find they can get fooled by flipping pixels! (I'll end my rant here with the caveat that I am a roboticist, so perhaps I have very skewed views of intelligence and feelings about embodied intelligence, but I'm willing to learn and be disabused of my notions!)
TheOtherHobbes 9y ago
Human object recognition has its own common failure modes. Take almost any photo that has dense content, mirror it, and "faces" or "creatures" will appear. We're primed to parse curvy symmetric shapes as organic, and if there's any hint at all of eyes we'll probably see a face. Optical illusions are a catalogue of other failures. We can easily be tricked into misreading colours, shapes, and lengths. Because the visual cortex works at a higher level of abstraction than naive pixel-reading classifiers, we're not likely to be confused by adversarial pixel flipping. But we're just as likely to misread equivalent abstractions at a higher level. Things get even worse at higher cognitive levels. The recent study into the spread of fake news shows that we're far more likely to mis-parse "facts" than to be ruthlessly objective. The underlying problem is that we expect AI to show the same predictability that simple digital logic does. AI won't and can't, because the problems operate on a completely different level and uncertainty is built into the domain. The best solutions will be better than humans, but it's naive to think infallibility is possible.
- roywiggins 9y ago
  It's true, we see faces where there aren't any. But we usually know very quickly that it's not a real person- even if we might be confused if we catch it out of the corner of our eye, as soon as we get a chance to look at it for more than a split second we will change our mind and not believe it's a real face. Of course there are more persistent failure modes where we can be tricked for a lot longer, and artificial illusions that persist despite knowing what they are.
pixelbash 9y ago
I think an interesting point here is that you often need to learn the distinguishing features themselves - especially when classifying quite similar things. as a child I found it hard to distinguish plants from each other, until I learned to look at features like flowers, leaf shape, bark, branch patterns etc.

ianai 9y ago

Yes you have a concept of an object based on all your memories across all senses. It takes all of your senses and their multiple sources to form concepts.

danbruc 9y ago
This is true insofar that the huge amount of background knowledge I have and use to build a model of what I see in an image was almost inevitably build over my lifetime with all my senses, but this does not change the fact that I performed the task at hand only visually, i.e. without fusing different senses. The AI is lacking my background knowledge, not senses other than vision. One might argue that building human-level background knowledge requires more than just vision but it seems at least not totally obvious that this is indeed the case.
TFortunato 9y ago
This is a very interesting point that I think has more going for it than the poster you are replying to may realize. Related, and apologies I don't have a link, but there have been a few stories circulated of adults who were blind at birth / a young age who had their sight restored at adulthood - They often had great difficulty, or even found it impossible to correlate what they saw with their eyes with the concept of objects that they knew by touch and their other senses.
- danbruc 9y ago
  I have no link at hand either, but I also read about blind people being unable to transfer their concepts of objects build from touching them to seeing them when their sight was restored. They know what a sharp edge or a pointy thing feels like but they don't know what it looks like. But I think that actually strengthens my point, blind people are capable of learning how different object feel and they recognize them this way but they so just fine without vision and if they suddenly gain access to vision it does not help them. In general having more senses will help to more reliably identify objects because you have access to more features to differentiate them, for example distinguishing materials from imitations just by vision can be hard or maybe even impossible. But when you can also touch them you gain a lot of new information about surface structure, hardness, thermal conductivity and so on and you can easily distinguish between, for example, real stone and plastic or wood with a stone print on it. But just because it is advantageous in the general case to have access to more than one sense that does not imply that it is necessary for a specific task.
  - TFortunato 9y ago
    Apologies ahead of time. I'm typing this on a small screen phone, and trying to not ramble :-) We might be more on the same page than I realized On the one hand, I agree that it doesn't imply that it is necessary to have access to more than one sense. On the other hand I do think that having access to more senses may make the problem a LOT easier, especially during the training process for teaching systems to do things like try to distinguish objects by sight. Don't get me wrong, the progress made so far has been incredible, but it isn't surprising that we are eventually running into limits given the limited amount of data these systems have to train on. Sort of riffing off of this - In particular, my sense is that when you are training on 2-D pixel data only, it is no wonder that no matter how clever your network is to extract high-level features from the data, you are going to run into these kind of issues. We are asking the system to perform a task (describing, classifying, etc.) about a fundamentally 3-d world, using only a 2-d image, when it has never had a "concept" of the world in 3 dimensions. I think that we take for granted that when we are learning, not only do we get information by seeing, but also touching and manipulating objects and existing in the world around us. We can see things in different lighting conditions, and we can also move our heads and our bodies and around the world and see things from new angles and manipulate them to learn the rich set of correlations between what we see, and what we experience...basically the rich structure of the world around us. A system which is trained on images alone doesn't have the integrated knowledge, which is what I was hoping to get at with the anecdote of adults being cured of blindness. When you look at some of the work we do to augment visual data sets, by scaling images, cropping them, rotating them, skewing them, etc. its basically a poor mimcry of something that humans and animals get just by being in the world...this idea of learning that things remain the same and have a certain structure, regardless of the viewpoint we see them from. (From what I understand, this idea is also part of what inspires capsule networks). Anyways, to come to my main point. Yes, you can absolutely identify objects, even a completely novel object after only seeing it once or twice. But I would argue that this is only because you already have a rich framework of the world as 3-dimensional spatially, along with all of the other priperties that object have that you have learned. So when you see a 2-dimensional picture, your brain can form a 3-d image of the scene, identify materials of objects, etc., which you can leverage. Things that our current AI systems have no idea of... so perhaps its a miracle they made it this far, and no wonder that we find they can get fooled by flipping pixels! (I'll end my rant here with the caveat that I am a roboticist, so perhaps I have very skewed views of intelligence and feelings about embodied intelligence, but I'm willing to learn and be disabused of my notions!)

TheOtherHobbes 9y ago

Human object recognition has its own common failure modes. Take almost any photo that has dense content, mirror it, and "faces" or "creatures" will appear. We're primed to parse curvy symmetric shapes as organic, and if there's any hint at all of eyes we'll probably see a face. Optical illusions are a catalogue of other failures. We can easily be tricked into misreading colours, shapes, and lengths. Because the visual cortex works at a higher level of abstraction than naive pixel-reading classifiers, we're not likely to be confused by adversarial pixel flipping. But we're just as likely to misread equivalent abstractions at a higher level. Things get even worse at higher cognitive levels. The recent study into the spread of fake news shows that we're far more likely to mis-parse "facts" than to be ruthlessly objective. The underlying problem is that we expect AI to show the same predictability that simple digital logic does. AI won't and can't, because the problems operate on a completely different level and uncertainty is built into the domain. The best solutions will be better than humans, but it's naive to think infallibility is possible.

roywiggins 9y ago
It's true, we see faces where there aren't any. But we usually know very quickly that it's not a real person- even if we might be confused if we catch it out of the corner of our eye, as soon as we get a chance to look at it for more than a split second we will change our mind and not believe it's a real face. Of course there are more persistent failure modes where we can be tricked for a lot longer, and artificial illusions that persist despite knowing what they are.

pixelbash 9y ago

I think an interesting point here is that you often need to learn the distinguishing features themselves - especially when classifying quite similar things. as a child I found it hard to distinguish plants from each other, until I learned to look at features like flowers, leaf shape, bark, branch patterns etc.