5 ms·

While Hinton's view need to be noted, I heard a quote attributed to Yann LeCun, something like, "If you want to learn flying by modeling the biology of birds,

While Hinton's view need to be noted, I heard a quote attributed to Yann LeCun, something like, "If you want to learn flying by modeling the biology of birds, you're doing it wrong. Just look at today's airplanes. They have no resemblance to birds at all. Yet they're million times better and faster than any bird."

bra-ket 6y ago
the problem with that analogy is trying to build an airplane before you figured out the laws of physics.
- hyko 6y ago
  People build things without understanding the underlying principles all the time, e.g. the steam engine. You could probably make the case that building things has helped our understanding more than our understanding has helped us build things. Having said that, you can certainly improve a design when you better understand the fundamentals (vs intuition + trial & error).
- sdenton4 6y ago
  The physics of lift and aerodynamics were faaaaar from well understood at the time of the first airplanes, though. New areas tend to run a bit ahead of the underlying science; the fundamentals expand to support and improve the applications over time.
  - bra-ket 6y ago
    but we did have quite a few advances at the time of the first airplane, for example by that time steam & combustion engines were already invented, which required non-trivial understanding of physics, chemistry and material science was very advanced. I hold a pessimistic view that we are still in hunter-gatherer mode as it comes to understanding cognition.
    - sdenton4 6y ago
      Well, it's your right to be a pessimist... I tend to think that the current hardware specialized for fast, parallelized linear algebra is at least as good as the wheels available at the start of the industrial revolution, though. We have learning algorithms that can match human/animal performance in a wide - but still constrained - set of tasks, which previous non-learned algorithms hadn't been able to crack. It's a start! At some point you have to strike rocks to make fire, because the butane lighter hasn't been invented yet. You make do with what's available, and progressively get better at it. I tend to think that we're a couple-few perspective shifts away from getting it 'right,' and that the hardware side likely barely matters. But, I'm an optimist.
justicezyx 6y ago
"They have no resemblance to birds at all. Yet they're million times better and faster than any bird." LMAO A 6 years old kid can see the fundamental resemblance between a bird and a modern passenger airplane: The wings Tail stabilizer Slender body Planes are faster bigger Are they better? Not necessarily, for example, humming bird can fly in a way that is far beyond any human machine in terms of efficiency and flexibility. Of course man should not imitate birds, because human flight is fundamentally different activity than bird flying. But to say human aviation did not start by mimicking birds, is like to say Ann was not inspired human brain...
- nightski 6y ago
  I think the point was that aviation at the time did start mimicking birds and that was why there was so much failure. It was not until they let go of mimicking birds and took a different approach that they found success.
- deehouie 6y ago
  See my link to his ICML talk above.
bbarnett 6y ago
I get what the author was trying to say, but it's still -- a very limited view. Mostly because of the last bit (better/faster). Birds are to planes, as humans are to cars. Yet can a car leap over barricades, climb mountains, trees, self-repair, turn on a dime, stop instantly, etc, etc? A plane cannot maneuver like a bird, take off in crazy weather conditions, land on a dime in a tree, stop almost instantly in flight, and change direction, etc. I think what you've quoted has a lot of value here, for, what we should expect from an artificial brain, isn't a human brain. This is truth. However, while it may be faster in a specific capacity, but it won't have the same characteristics. So yes, expecting it to be like a human brain doesn't make sense. Yet better/faster? I don't think we can compare this, they're too different. (which is really the quote's point, but I just didn't like the better/faster bit at the end...)
- nn3 6y ago
  Also birds (and insects/bats/pterosaurs) flight is a lot more energy efficient than any plane. Today's deep learning is essentially brute force, burning thousands of watts for anything more complicated which a single human brain can often do in ~15Watts. The advanced models like GPT-3 are burning millions of watts in the cloud but they're not that much better than what a brain can do (and in many ways worse, as in often requiring supervised learning) That's the key point. The algorithms need to become more energy efficient to make significant leaps, thus become more like brains.
  - ant6n 6y ago
    Also, birds produce themselves out of an egg, with only food, water and air as production input. They also can produce more of themselves with minimal input. They are also self-repairing/maintaining, something planes cant do.
  - starpilot 6y ago
    No, it's not. What's your comparison? Are there birds that can carry 80,000 lb of passenger + cargo weight? Condors fly like fixed-wing aircraft for 99% of their flight, hummingbirds fly more like insects. There isn't one type of bird flight. This whole HN discussion of bird flight is a trainwreck and reflects massive gaps in understanding of aerodynamics. This is '00s "computer virus news report" level competence in this subject.
habitue 6y ago
It's true we don't build planes like birds. But we understand how bird wings work, and we understand why we don't design planes like bird wings. We don't have the same kind of understanding of how brains learn, so the comparison is not quite right. When we understand how to build things that learn like brains, we'll be in a better position to say things like "Ok this is strictly worse than backprop, let's stick with backprop" or "Actually, this is better than backprop because X", (or, more likely, there are things we can use from both). Until we have that understanding it's silly to stop trying to understand how the brain does things. That being said, nobody is going to stop working on backprop, and no one is going to stop working on understanding biological mechanisms . Research works by a bunch of people investigating different avenues simultaneously.
cbarrick 6y ago
Reminds me of Dijkstra's aphorism: > "The question of whether machines can think is about as relevant as the question of whether submarines can swim."
deehouie 6y ago
I found the source. He and Ranzato gave a talk at ICML 2013 https://cilvr.nyu.edu/lib/exe/fetch.php?media=deeplearning:2016:dl-convnets.pdf https://cilvr.nyu.edu/lib/exe/fetch.php?media=deeplearning:2... Slide #9 Let's be inspired by nature, but not too much It's nice imitate Nature, But we also need to understand For airplanes, we developed aerodynamics and compressible fluid dynamics. Question : what is the equivalent of aerodynamics for understanding intelligence
JohnJamesRambo 6y ago
I would caveat "million times better." When 747s pop out baby 747s and procreate and refuel autonomously for 60 millon years I will give you million times better. But I like the gist of the quote.
jokethrowaway 6y ago
Better is subjective. A specific AI may perform better than a human (eg. processing images for hours, faster and longer than a human) But we're still not able to have an AI you can talk to like you talk to a human (gpt-3 is still nowhere enough in my book, albeit being a good technological feat)

bra-ket 6y ago

the problem with that analogy is trying to build an airplane before you figured out the laws of physics.

hyko 6y ago
People build things without understanding the underlying principles all the time, e.g. the steam engine. You could probably make the case that building things has helped our understanding more than our understanding has helped us build things. Having said that, you can certainly improve a design when you better understand the fundamentals (vs intuition + trial & error).
sdenton4 6y ago
The physics of lift and aerodynamics were faaaaar from well understood at the time of the first airplanes, though. New areas tend to run a bit ahead of the underlying science; the fundamentals expand to support and improve the applications over time.
- bra-ket 6y ago
  but we did have quite a few advances at the time of the first airplane, for example by that time steam & combustion engines were already invented, which required non-trivial understanding of physics, chemistry and material science was very advanced. I hold a pessimistic view that we are still in hunter-gatherer mode as it comes to understanding cognition.
  - sdenton4 6y ago
    Well, it's your right to be a pessimist... I tend to think that the current hardware specialized for fast, parallelized linear algebra is at least as good as the wheels available at the start of the industrial revolution, though. We have learning algorithms that can match human/animal performance in a wide - but still constrained - set of tasks, which previous non-learned algorithms hadn't been able to crack. It's a start! At some point you have to strike rocks to make fire, because the butane lighter hasn't been invented yet. You make do with what's available, and progressively get better at it. I tend to think that we're a couple-few perspective shifts away from getting it 'right,' and that the hardware side likely barely matters. But, I'm an optimist.

justicezyx 6y ago

"They have no resemblance to birds at all. Yet they're million times better and faster than any bird." LMAO A 6 years old kid can see the fundamental resemblance between a bird and a modern passenger airplane: The wings Tail stabilizer Slender body Planes are faster bigger Are they better? Not necessarily, for example, humming bird can fly in a way that is far beyond any human machine in terms of efficiency and flexibility. Of course man should not imitate birds, because human flight is fundamentally different activity than bird flying. But to say human aviation did not start by mimicking birds, is like to say Ann was not inspired human brain...

nightski 6y ago
I think the point was that aviation at the time did start mimicking birds and that was why there was so much failure. It was not until they let go of mimicking birds and took a different approach that they found success.
deehouie 6y ago
See my link to his ICML talk above.

bbarnett 6y ago

I get what the author was trying to say, but it's still -- a very limited view. Mostly because of the last bit (better/faster). Birds are to planes, as humans are to cars. Yet can a car leap over barricades, climb mountains, trees, self-repair, turn on a dime, stop instantly, etc, etc? A plane cannot maneuver like a bird, take off in crazy weather conditions, land on a dime in a tree, stop almost instantly in flight, and change direction, etc. I think what you've quoted has a lot of value here, for, what we should expect from an artificial brain, isn't a human brain. This is truth. However, while it may be faster in a specific capacity, but it won't have the same characteristics. So yes, expecting it to be like a human brain doesn't make sense. Yet better/faster? I don't think we can compare this, they're too different. (which is really the quote's point, but I just didn't like the better/faster bit at the end...)

nn3 6y ago
Also birds (and insects/bats/pterosaurs) flight is a lot more energy efficient than any plane. Today's deep learning is essentially brute force, burning thousands of watts for anything more complicated which a single human brain can often do in ~15Watts. The advanced models like GPT-3 are burning millions of watts in the cloud but they're not that much better than what a brain can do (and in many ways worse, as in often requiring supervised learning) That's the key point. The algorithms need to become more energy efficient to make significant leaps, thus become more like brains.
- ant6n 6y ago
  Also, birds produce themselves out of an egg, with only food, water and air as production input. They also can produce more of themselves with minimal input. They are also self-repairing/maintaining, something planes cant do.
- starpilot 6y ago
  No, it's not. What's your comparison? Are there birds that can carry 80,000 lb of passenger + cargo weight? Condors fly like fixed-wing aircraft for 99% of their flight, hummingbirds fly more like insects. There isn't one type of bird flight. This whole HN discussion of bird flight is a trainwreck and reflects massive gaps in understanding of aerodynamics. This is '00s "computer virus news report" level competence in this subject.

habitue 6y ago

It's true we don't build planes like birds. But we understand how bird wings work, and we understand why we don't design planes like bird wings. We don't have the same kind of understanding of how brains learn, so the comparison is not quite right. When we understand how to build things that learn like brains, we'll be in a better position to say things like "Ok this is strictly worse than backprop, let's stick with backprop" or "Actually, this is better than backprop because X", (or, more likely, there are things we can use from both). Until we have that understanding it's silly to stop trying to understand how the brain does things. That being said, nobody is going to stop working on backprop, and no one is going to stop working on understanding biological mechanisms . Research works by a bunch of people investigating different avenues simultaneously.

cbarrick 6y ago

Reminds me of Dijkstra's aphorism: > "The question of whether machines can think is about as relevant as the question of whether submarines can swim."

deehouie 6y ago

I found the source. He and Ranzato gave a talk at ICML 2013 https://cilvr.nyu.edu/lib/exe/fetch.php?media=deeplearning:2016:dl-convnets.pdf https://cilvr.nyu.edu/lib/exe/fetch.php?media=deeplearning:2... Slide #9 Let's be inspired by nature, but not too much It's nice imitate Nature, But we also need to understand For airplanes, we developed aerodynamics and compressible fluid dynamics. Question : what is the equivalent of aerodynamics for understanding intelligence

JohnJamesRambo 6y ago

I would caveat "million times better." When 747s pop out baby 747s and procreate and refuel autonomously for 60 millon years I will give you million times better. But I like the gist of the quote.

jokethrowaway 6y ago

Better is subjective. A specific AI may perform better than a human (eg. processing images for hours, faster and longer than a human) But we're still not able to have an AI you can talk to like you talk to a human (gpt-3 is still nowhere enough in my book, albeit being a good technological feat)