4 ms·

That is the problem with generalization and cop outs like these. It's no good to people in the field doing actual work where the devil is in the detail. Big da

That is the problem with generalization and cop outs like these. It's no good to people in the field doing actual work where the devil is in the detail. Big data is fairly important to a lot of things, for example I was listening to Tesla's use of Deep net models where they mentioned that there were literally so many variations of Stop Signs that they needed to learn what was really in the "tail" of the distribution of Stop Sign types to construct reliable AI

vasco 5y ago
Interestingly, when you learn how to drive you need to see approximately one example and you're able to identify them all.
- corndoge 5y ago
  Is there some underlying point to this statement? It comes off as a passive dismissal of something but I'm not sure what. It might be helpful to directly state what you're trying to say so that other people can engage with it.
  - vasco 5y ago
    It doesn't seem like the other replies are having trouble engaging with it. Since we're giving each other advice, you should use your down votes instead of pontificating that other people's comments are "passive dismissals" when you don't like or understand them.
- teruakohatu 5y ago
  That is called transfer learning. You might only need to see one photo of a sign to identify it in real life (although arguably learner drivers take a while to notice signs) but that is only because you have been training on identifying generic objects since you left the womb. You brain already knows how to select the most important features of a sign. The shape, the size and the color. You have also learned how to understand the text on the sign. A new born baby does not have that ability. This is applied in ANN as well. Transfer learning is using a pre-trained neural network, which has already learned identifying objects, and then using it to train on identifying a new, usually smaller, set of objects using, usually, a lot less training data. That is what Andrew is talking about in the article.
  - gary_0 5y ago
    > The shape, the size and the color. And the context. For example, self-driving cars need to account for "Pizza Stop" restaurant signage, placards stuck to telephone poles that say things like "Stop Cancer", stop signs retracted into the sides of school buses, signs with additional instructions like "Stop when lights flashing", road workers with handheld stop signs, and the unconventional stop signs you see in parking lots. You can probably get pretty far by checking the proximity to the road, height, dimensions, orientation, what it's mounted on, and if the sign incorporates any other text. But you can't just scan some pixels for "red octagon with STOP on it".
  - extheat 5y ago
    I'm not sure "training" is the right way to think about it. Children don't train to identify objects, they quickly develop the ability to recognize objects and are able to correlate them with prior information that was retained and learned. Case in point: if you take a child born blind, give them the ability to see, they are immediately able to recognize and correlate objects around them.
    - teruakohatu 5y ago
      > Children don't train to identify objects I have to disagree. They spend an inordinate amount of time trying to understand what they see, taste and hear. > able to correlate them with prior information that was retained and learned This is what we call inference. > Case in point: if you take a child born blind, give them the ability to see, they are immediately able to recognize and correlate objects around them. Not everyone who is legally blind can see absolutely nothing, but people who have recovered from complete vision loss [1] have problems. Mike May [2] lost vision as 3 year old child and regained it in his 40s. Despite seeing for the first three years of his life, years after regaining vision he was unable to see in 3D or recognize people from faces alone. Blind people do not lack spatial awareness, so being able to recognize objects with context if they regained sight with would not surprise me. There are blind people that can "see" with echo location using parts of the brain associated with visual processing [3] But for example in Mike's case, he was unable to recognize close family by their faces years after regaining vision, he needed additional context. Many things we take for granted as being innate to the human experience, are in fact learned (trained) behavior. [1] https://en.wikipedia.org/wiki/Recovery_from_blindness https://en.wikipedia.org/wiki/Recovery_from_blindness [2] https://en.wikipedia.org/wiki/Mike_May_(skier) https://en.wikipedia.org/wiki/Mike_May_(skier) [3] https://en.wikipedia.org/wiki/Human_echolocation https://en.wikipedia.org/wiki/Human_echolocation
- itissid 5y ago
  It does feel though that a model like the human mind will be very fundamentally different from any of the models of today. No? Like the NN State of the art models of today are so different from state of the art 12 or so years ago which was SVMs.
- onethought 5y ago
  when you learn to drive you need [approximately 16 years training your visual system hardware that took 6 million years to evolve and] to see approximately one example and you're able to identify them all. FTFY. Yet Tesla have been working on both the hardware and software for 10 years? Amazing progress right?
  - deleted 5y ago
    [deleted]
  - nuclearnice1 5y ago
    If you’re going back 6 million years on evolution, would it make sense to push back the start date for Tesla’s work as well? It seems the right parallel to evolution might include predecessor inventions in vision, computation, and beyond.
    - onethought 5y ago
      Sure… so 100 years? 2000? Take your pick. AI Vision is developing ridiculously fast.
- simulate-me 5y ago
  Your brain is also the result of billions of years of evolutionary "training." Neural nets start from scratch.
IIAOPSW 5y ago
Sounds like they missed the forest and instead "deep learned" all the variations of trees.
bennyg 5y ago
Do you have a link for that Tesla talk?

vasco 5y ago

Interestingly, when you learn how to drive you need to see approximately one example and you're able to identify them all.

corndoge 5y ago
Is there some underlying point to this statement? It comes off as a passive dismissal of something but I'm not sure what. It might be helpful to directly state what you're trying to say so that other people can engage with it.
- vasco 5y ago
  It doesn't seem like the other replies are having trouble engaging with it. Since we're giving each other advice, you should use your down votes instead of pontificating that other people's comments are "passive dismissals" when you don't like or understand them.
teruakohatu 5y ago
That is called transfer learning. You might only need to see one photo of a sign to identify it in real life (although arguably learner drivers take a while to notice signs) but that is only because you have been training on identifying generic objects since you left the womb. You brain already knows how to select the most important features of a sign. The shape, the size and the color. You have also learned how to understand the text on the sign. A new born baby does not have that ability. This is applied in ANN as well. Transfer learning is using a pre-trained neural network, which has already learned identifying objects, and then using it to train on identifying a new, usually smaller, set of objects using, usually, a lot less training data. That is what Andrew is talking about in the article.
- gary_0 5y ago
  > The shape, the size and the color. And the context. For example, self-driving cars need to account for "Pizza Stop" restaurant signage, placards stuck to telephone poles that say things like "Stop Cancer", stop signs retracted into the sides of school buses, signs with additional instructions like "Stop when lights flashing", road workers with handheld stop signs, and the unconventional stop signs you see in parking lots. You can probably get pretty far by checking the proximity to the road, height, dimensions, orientation, what it's mounted on, and if the sign incorporates any other text. But you can't just scan some pixels for "red octagon with STOP on it".
- extheat 5y ago
  I'm not sure "training" is the right way to think about it. Children don't train to identify objects, they quickly develop the ability to recognize objects and are able to correlate them with prior information that was retained and learned. Case in point: if you take a child born blind, give them the ability to see, they are immediately able to recognize and correlate objects around them.
  - teruakohatu 5y ago
    > Children don't train to identify objects I have to disagree. They spend an inordinate amount of time trying to understand what they see, taste and hear. > able to correlate them with prior information that was retained and learned This is what we call inference. > Case in point: if you take a child born blind, give them the ability to see, they are immediately able to recognize and correlate objects around them. Not everyone who is legally blind can see absolutely nothing, but people who have recovered from complete vision loss [1] have problems. Mike May [2] lost vision as 3 year old child and regained it in his 40s. Despite seeing for the first three years of his life, years after regaining vision he was unable to see in 3D or recognize people from faces alone. Blind people do not lack spatial awareness, so being able to recognize objects with context if they regained sight with would not surprise me. There are blind people that can "see" with echo location using parts of the brain associated with visual processing [3] But for example in Mike's case, he was unable to recognize close family by their faces years after regaining vision, he needed additional context. Many things we take for granted as being innate to the human experience, are in fact learned (trained) behavior. [1] https://en.wikipedia.org/wiki/Recovery_from_blindness https://en.wikipedia.org/wiki/Recovery_from_blindness [2] https://en.wikipedia.org/wiki/Mike_May_(skier) https://en.wikipedia.org/wiki/Mike_May_(skier) [3] https://en.wikipedia.org/wiki/Human_echolocation https://en.wikipedia.org/wiki/Human_echolocation
itissid 5y ago
It does feel though that a model like the human mind will be very fundamentally different from any of the models of today. No? Like the NN State of the art models of today are so different from state of the art 12 or so years ago which was SVMs.
onethought 5y ago
when you learn to drive you need [approximately 16 years training your visual system hardware that took 6 million years to evolve and] to see approximately one example and you're able to identify them all. FTFY. Yet Tesla have been working on both the hardware and software for 10 years? Amazing progress right?
- deleted 5y ago
  [deleted]
- nuclearnice1 5y ago
  If you’re going back 6 million years on evolution, would it make sense to push back the start date for Tesla’s work as well? It seems the right parallel to evolution might include predecessor inventions in vision, computation, and beyond.
  - onethought 5y ago
    Sure… so 100 years? 2000? Take your pick. AI Vision is developing ridiculously fast.
simulate-me 5y ago
Your brain is also the result of billions of years of evolutionary "training." Neural nets start from scratch.

IIAOPSW 5y ago

Sounds like they missed the forest and instead "deep learned" all the variations of trees.

bennyg 5y ago

Do you have a link for that Tesla talk?