5 ms·

> People drive reasonably well using vision primarily This is not accurate however. Other important senses in use include proprioceptive, hearing and tactile f

by Cybiote 10y ago

> People drive reasonably well using vision primarily This is not accurate however. Other important senses in use include proprioceptive, hearing and tactile feedback from wheels. In addition to vision and the improved dynamic range of eyes, there is the important fact that human vision integrates a world model into expectations. Human vision also models time and motion which help manage where to focus attention. Humans can additionally predict other agents and other things about the world based on intuitive physics. This is why they can get on without the huge array of sensors and cars cannot. Humans make up for the lack of sensors by being able to use the poor quality data more effectively. To put this in perspective, 8.75 megabits / second is estimated to pass through the human retina but only on the order of a 100 bits is estimated to reach conscious attention. > Computer learning networks can classify imagery at least as accurately as humans and sometimes more so. This is true but only in a limited sense. For example, when I put in the image on the right (of a car in a swimming pool) from http://icml.cc/2015/invited/LeonBottouICML2015.pdf#page=58 http://icml.cc/2015/invited/LeonBottouICML2015.pdf#page=58 (which you should read and find the talk of but) in ResNet I get as top results: 0.2947; screen, CRT screen golfcart, golf cart boathouse amphibian, amphibious vehicle For LeNet it's: 0.5422; amphibian, amphibious vehicle jeep, landrover wreck speedboat The key difference is learning in animals occurs by breaking things down in terms of modular concepts, so even when things are not recognized new things can be labeled as a composition of smaller nearby concepts. Machines cannot yet do this well at all and certainly not as flexibly. Things as lighting and shading do not move animals as much in the concept space. > The execution strategy appears to be to run classification and command prediction all the time, and while the human is in control consider it supervised learning. This strategy will not learn from accidents because the signal there will be far from optimal usually.

dave_sullivan 10y ago
> The key difference is learning in animals occurs by breaking things down in terms of modular concepts, so even when things are not recognized new things can be labeled as a composition of smaller nearby concepts. Machines cannot yet do this well at all and certainly not as flexibly. Actually, that's pretty much what deep learning is doing. For instance: https://papers.nips.cc/paper/5027-zero-shot-learning-through-cross-modal-transfer https://papers.nips.cc/paper/5027-zero-shot-learning-through... That paper was from a few years ago, I think the state of the art is better now, but it's trying to do exactly what you're talking about. More broadly, what you're talking about falls under the umbrella of transfer learning (that is, a model's ability to learn helpful information about task Y by training on related task X, preferably by learning and sharing useful features.)
- Hydraulix989 10y ago
  Well, not with traditional feedforward networks (LeNet, etc.). You can't run the classifier and find tires, then wheels, and then a car; but you do get composition of features.
  - dave_sullivan 10y ago
    > not with traditional feedforward networks (LeNet, etc.) I'd argue they are implicitly doing this. > You can't run the classifier and find tires, then wheels, and then a car; Why can't you run a classifier for tires, one for wheels, one for cars, then combine their outputs for a final classifier maybe based on a decision tree? You can train all the networks at the same time and it will give you a probability distributions for all 4 outputs (tires, wheels, cars, blended). What am I missing?
    - Hydraulix989 10y ago
      Doesn't scale. You don't have N brains to compose every representation.
    - argonaut 10y ago
      That would just be your opinion. It has not been shown. It's still an open research question over what neural nets are actually learning in their intermediate layers. You're going to need large amounts of fine-grained labeled data for each category. You've also just manually determined some sort of (brittle) object ontology. What if there are only 3 tires? What if there are four tires on the road but no car? All sorts of edge cases, and all you've done is train a classifier for cars, not actually solved driving in any meaningful way.
  - pakl 10y ago
    Agreed -- If you get away from traditional feedforward networks by adding recurrence throughout, then at least there is some chance of learning scale-free features and compositionality.
- Cybiote 10y ago
enraged_camel 10y ago
>>This is not accurate however. Other important senses in use include proprioceptive, hearing and tactile feedback from wheels. In addition to vision and the improved dynamic range of eyes, there is the important fact that human vision integrates a world model into expectations. Human vision also models time and motion which help manage where to focus attention. Humans can additionally predict other agents and other things about the world based on intuitive physics. This is why they can get on without the huge array of sensors and cars cannot. Humans make up for the lack of sensors by being able to use the poor quality data more effectively. Yes, but all those things you listed are very imperfect and can increase the risk of mistakes. For example, someone two lanes over honking their horn can cause a momentary distraction for you ("are they honking at me?"), causing you to not notice a cyclist cutting in front of you. And so can a dancing clown on the sidewalk.
- Cybiote 10y ago
  Very much so and that's why we don't want our cars to perfectly replicate humans. More sensors plus a limited and narrow AI is better than just vision and a smarter AI.
viraptor 10y ago
> This is true but only in a limited sense. Isn't what we need very limited anyway? What the cars need is recognition of obstacles and the type of the obstacle in a very limited range. Basically when something is on the road, it doesn't matter whether it's a moose or a deer - you slow down and avoid, or brake depending on the environment. "what is in this picture" classifiers don't seem like a good algorithm to use in that case. Object detection / feature extraction seems to be much closer.
- kpil 10y ago
  Eh. Is it clouds or a truck, or a shadow on the road or a moose? A tree besides the road or a cyclist) Hard to tell without really good classifiers. I guess that with a lidar or some kind of 3d perception, you can relax the demands on the classifier a bit and ignore really flat things and the sky...
- argonaut 10y ago
  Object detection is considered a more difficult problem than image classification in the vision community. So I'm not sure what the point is there.
pakl 10y ago
I commend you. Very few people actually try feeding images into these things to show how bad they are. For a real shock at how poor performance is, try feeding frames of video in. (Video is different because it generally doesn't have carefully stereotypically framed, exposed, in-focus content). Edit: examples of ResNet applied to video by a colleague http://blog.piekniewski.info/2016/08/12/how-close-are-we-to-vision/ http://blog.piekniewski.info/2016/08/12/how-close-are-we-to-...
- Capt-RogerOver 10y ago
  Very informative quote: So where do the reports of superhuman abilities come from? Well since there are many breeds of dogs in the ImageNet, an average human (like me) will not be able to distinguish half of them (say Staffordshire bullterrier from Irish terrier or English foxhound - yes there are real categories in ImageNet, believe it or not). The network which was "trained to death" on this dataset will obviously be better at that aspect. In all practical aspects an average human (even a child) is orders of magnitude better at understanding/describing scenes than the best deep nets (as of late 2015) trained on ImageNet.

by Cybiote 10y ago

dave_sullivan 10y ago

> The key difference is learning in animals occurs by breaking things down in terms of modular concepts, so even when things are not recognized new things can be labeled as a composition of smaller nearby concepts. Machines cannot yet do this well at all and certainly not as flexibly. Actually, that's pretty much what deep learning is doing. For instance: https://papers.nips.cc/paper/5027-zero-shot-learning-through-cross-modal-transfer https://papers.nips.cc/paper/5027-zero-shot-learning-through... That paper was from a few years ago, I think the state of the art is better now, but it's trying to do exactly what you're talking about. More broadly, what you're talking about falls under the umbrella of transfer learning (that is, a model's ability to learn helpful information about task Y by training on related task X, preferably by learning and sharing useful features.)

Hydraulix989 10y ago
Well, not with traditional feedforward networks (LeNet, etc.). You can't run the classifier and find tires, then wheels, and then a car; but you do get composition of features.
- dave_sullivan 10y ago
  > not with traditional feedforward networks (LeNet, etc.) I'd argue they are implicitly doing this. > You can't run the classifier and find tires, then wheels, and then a car; Why can't you run a classifier for tires, one for wheels, one for cars, then combine their outputs for a final classifier maybe based on a decision tree? You can train all the networks at the same time and it will give you a probability distributions for all 4 outputs (tires, wheels, cars, blended). What am I missing?
  - Hydraulix989 10y ago
    Doesn't scale. You don't have N brains to compose every representation.
  - argonaut 10y ago
    That would just be your opinion. It has not been shown. It's still an open research question over what neural nets are actually learning in their intermediate layers. You're going to need large amounts of fine-grained labeled data for each category. You've also just manually determined some sort of (brittle) object ontology. What if there are only 3 tires? What if there are four tires on the road but no car? All sorts of edge cases, and all you've done is train a classifier for cars, not actually solved driving in any meaningful way.
- pakl 10y ago
  Agreed -- If you get away from traditional feedforward networks by adding recurrence throughout, then at least there is some chance of learning scale-free features and compositionality.
Cybiote 10y ago

enraged_camel 10y ago

>>This is not accurate however. Other important senses in use include proprioceptive, hearing and tactile feedback from wheels. In addition to vision and the improved dynamic range of eyes, there is the important fact that human vision integrates a world model into expectations. Human vision also models time and motion which help manage where to focus attention. Humans can additionally predict other agents and other things about the world based on intuitive physics. This is why they can get on without the huge array of sensors and cars cannot. Humans make up for the lack of sensors by being able to use the poor quality data more effectively. Yes, but all those things you listed are very imperfect and can increase the risk of mistakes. For example, someone two lanes over honking their horn can cause a momentary distraction for you ("are they honking at me?"), causing you to not notice a cyclist cutting in front of you. And so can a dancing clown on the sidewalk.

Cybiote 10y ago
Very much so and that's why we don't want our cars to perfectly replicate humans. More sensors plus a limited and narrow AI is better than just vision and a smarter AI.

viraptor 10y ago

> This is true but only in a limited sense. Isn't what we need very limited anyway? What the cars need is recognition of obstacles and the type of the obstacle in a very limited range. Basically when something is on the road, it doesn't matter whether it's a moose or a deer - you slow down and avoid, or brake depending on the environment. "what is in this picture" classifiers don't seem like a good algorithm to use in that case. Object detection / feature extraction seems to be much closer.

kpil 10y ago
Eh. Is it clouds or a truck, or a shadow on the road or a moose? A tree besides the road or a cyclist) Hard to tell without really good classifiers. I guess that with a lidar or some kind of 3d perception, you can relax the demands on the classifier a bit and ignore really flat things and the sky...
argonaut 10y ago
Object detection is considered a more difficult problem than image classification in the vision community. So I'm not sure what the point is there.

pakl 10y ago

I commend you. Very few people actually try feeding images into these things to show how bad they are. For a real shock at how poor performance is, try feeding frames of video in. (Video is different because it generally doesn't have carefully stereotypically framed, exposed, in-focus content). Edit: examples of ResNet applied to video by a colleague http://blog.piekniewski.info/2016/08/12/how-close-are-we-to-vision/ http://blog.piekniewski.info/2016/08/12/how-close-are-we-to-...

Capt-RogerOver 10y ago
Very informative quote: So where do the reports of superhuman abilities come from? Well since there are many breeds of dogs in the ImageNet, an average human (like me) will not be able to distinguish half of them (say Staffordshire bullterrier from Irish terrier or English foxhound - yes there are real categories in ImageNet, believe it or not). The network which was "trained to death" on this dataset will obviously be better at that aspect. In all practical aspects an average human (even a child) is orders of magnitude better at understanding/describing scenes than the best deep nets (as of late 2015) trained on ImageNet.