5 ms·

For a more balanced view of the history of deep learning see Schmidhuber‘s review http://people.idsia.ch/~juergen/deep-learning-overview.html http://people.idsi

For a more balanced view of the history of deep learning see Schmidhuber‘s review http://people.idsia.ch/~juergen/deep-learning-overview.html http://people.idsia.ch/~juergen/deep-learning-overview.html and http://people.idsia.ch/~juergen/who-invented-backpropagation.html http://people.idsia.ch/~juergen/who-invented-backpropagation....

bobosha 8y ago
No invention occurs in a vacuum, most creations are recombinant effects.The pieces of the puzzle might very well have existed, but Hinton deservedly receives credit for putting the various pieces together for backprop IMHO.
- deleted 8y ago
  [deleted]
- obviuosly 8y ago
  Various researchers, e.g. Werbos, Linnainmaa, Bryson and Yu-Chi Ho have been doing backprop in neural networks before Hinton et al. Hinton was merely a popularizer of the idea.
  - nabla9 8y ago
    Wright brothers were most likely not the first to achieve heavier-than-air powered flight. But they made the first controlled, sustained flight. Controlled flight was the real breakthrough in aviation. It's the same with Hinton et. all. They were among many other pioneers, but what really sets them apart is that they made it all work. They also analyzed why it works.
    - kodz4 8y ago
      Octave Chanute in the case of heavier than air flight. He was the spark and the hub that connected most of the pioneers. https://en.wikipedia.org/wiki/Octave_Chanute https://en.wikipedia.org/wiki/Octave_Chanute
    - lettergram 8y ago
      For reference: https://en.m.wikipedia.org/wiki/Gustave_Whitehead https://en.m.wikipedia.org/wiki/Gustave_Whitehead Ive seen some debate around Gustave Whitehead. There apparently is some evidence that he achieved powered flight as early as 1899. It is claimed, he did not speak English very well. So it’s suspected he just didn’t publicize his work as much. Replicas of his early flying machine designs have been shown to fly. Including one from 1901: https://en.m.wikipedia.org/wiki/Whitehead_No._21 https://en.m.wikipedia.org/wiki/Whitehead_No._21
    - simonster 8y ago
      Schmidhuber's take on the Wright brothers: https://www.nature.com/articles/421689c https://www.nature.com/articles/421689c https://www.nature.com/articles/d41586-019-00491-5 https://www.nature.com/articles/d41586-019-00491-5
      - jessriedel 8y ago
        > In 1890 Clément Ader made the first manned, powered, heavier-than-air flight, of 50 m, in his bat-winged monoplane. Seems disingenuous to not mention that this device only worked due to ground effects and "flew" 8 inches off the ground (according to Wikipedia).
    - obviuosly 8y ago
      > but what really sets them apart is that they made it all work [citation needed] Werbos and others were very aware that the method worked and also of the implications of it: > This approach makes it possible to develop generalized, adaptive artificial intelligence, capable of achieving results comparable to what is discussed in science fiction http://werbos.com/Neural/SensitivityIFIPSeptember1981.pdf http://werbos.com/Neural/SensitivityIFIPSeptember1981.pdf The Hinton cartel happened to be in the right condition that allowed them to ignore the politically correct BS starting at the turn of the century which rendered the study of neurons, IQ and intelligence massively unpopular; and ruthless enough to only cite one another and call themselves the "fathers of deep learning" rather than actually citing the originators of these ideas.
  - backpropaganda 8y ago
    "merely" People deserve credit for taking an idea and running with it. Without Hinton's efforts (in 1987, 2006, or 2012), deep learning would have never seen the explosive growth.
    - srean 8y ago
      But is PR what Turing award is for though ?
      - mostafab 8y ago
        PR is what awards are all about
        backpropaganda 8y ago
        PR is what all science is about. Without communicating ideas, science is not useful. The trio above managed to communicate deep learning ideas to the wider vision and NLP community such as by collaborating with them to come up with benchmarks which would prove usefulness of deep learning, but also to industry and general public. Schmidhuber did very little on these fronts, and even his technical contributions while laudable do not compare with the awardee's.
    - obviuosly 8y ago
      > People deserve credit for taking an idea and running with it. Yes, as popularizers. > deep learning would have never seen the explosive growth This is extremely unlikely to be true. I think the explosive growth occured because neurons and intelligence were unpopular in the 2000s because of PC, deferring progress in this area to occur explosively later. The Hinton cartel were not the only people who were aware that AI research "has become silly" (in his own words).
    - vbuwivbiu 8y ago
      "would have never..." it's not possible to make such an assertion unless you have access to at least one copy of this universe that doesn't include him. Maybe in all the other universes the exact same thing happened except someone else did it. Maybe in some of those universes, without him and his popularizing of this technique, some other better technique flourished. Who knows? Stating "would never have" is nothing more than telling a story.
tassl 8y ago
Direct link with the pdf version: http://people.idsia.ch/~juergen/DeepLearning2July2014.pdf http://people.idsia.ch/~juergen/DeepLearning2July2014.pdf Unfortunately, a good PR is definitely important for prizes.
freyir 8y ago
I could imagine this announcement will drive Schmidhuber up the wall. He's been very outspoken about how credit in the field should be assigned. That said, I don't know how you could argue against Hinton's impact on the field, or LeCun and Bengio's achievements.
dkural 8y ago
Fascinating! Thank you for sharing. The issues around the assignment of credit is as old as science and the concept of discovery; reading "The Invention of Science: A New History of the Scientific Revolution" by David Wootton really opened my eyes into how this is actually a fundamental feature of our modern conception of scientific discovery (sounds counter-intuitive.. I know!). It's like a societal "bug".
hobo_mark 8y ago
Bloomberg podcast about Schmidhuber, which was also quite entertaining: https://m.soundcloud.com/bloomberg-business/deeper-minds-read-aloud https://m.soundcloud.com/bloomberg-business/deeper-minds-rea...

bobosha 8y ago

No invention occurs in a vacuum, most creations are recombinant effects.The pieces of the puzzle might very well have existed, but Hinton deservedly receives credit for putting the various pieces together for backprop IMHO.

deleted 8y ago
[deleted]
obviuosly 8y ago
Various researchers, e.g. Werbos, Linnainmaa, Bryson and Yu-Chi Ho have been doing backprop in neural networks before Hinton et al. Hinton was merely a popularizer of the idea.
- nabla9 8y ago
  Wright brothers were most likely not the first to achieve heavier-than-air powered flight. But they made the first controlled, sustained flight. Controlled flight was the real breakthrough in aviation. It's the same with Hinton et. all. They were among many other pioneers, but what really sets them apart is that they made it all work. They also analyzed why it works.
  - kodz4 8y ago
    Octave Chanute in the case of heavier than air flight. He was the spark and the hub that connected most of the pioneers. https://en.wikipedia.org/wiki/Octave_Chanute https://en.wikipedia.org/wiki/Octave_Chanute
  - lettergram 8y ago
    For reference: https://en.m.wikipedia.org/wiki/Gustave_Whitehead https://en.m.wikipedia.org/wiki/Gustave_Whitehead Ive seen some debate around Gustave Whitehead. There apparently is some evidence that he achieved powered flight as early as 1899. It is claimed, he did not speak English very well. So it’s suspected he just didn’t publicize his work as much. Replicas of his early flying machine designs have been shown to fly. Including one from 1901: https://en.m.wikipedia.org/wiki/Whitehead_No._21 https://en.m.wikipedia.org/wiki/Whitehead_No._21
  - simonster 8y ago
    Schmidhuber's take on the Wright brothers: https://www.nature.com/articles/421689c https://www.nature.com/articles/421689c https://www.nature.com/articles/d41586-019-00491-5 https://www.nature.com/articles/d41586-019-00491-5
    - jessriedel 8y ago
      > In 1890 Clément Ader made the first manned, powered, heavier-than-air flight, of 50 m, in his bat-winged monoplane. Seems disingenuous to not mention that this device only worked due to ground effects and "flew" 8 inches off the ground (according to Wikipedia).
  - obviuosly 8y ago
    > but what really sets them apart is that they made it all work [citation needed] Werbos and others were very aware that the method worked and also of the implications of it: > This approach makes it possible to develop generalized, adaptive artificial intelligence, capable of achieving results comparable to what is discussed in science fiction http://werbos.com/Neural/SensitivityIFIPSeptember1981.pdf http://werbos.com/Neural/SensitivityIFIPSeptember1981.pdf The Hinton cartel happened to be in the right condition that allowed them to ignore the politically correct BS starting at the turn of the century which rendered the study of neurons, IQ and intelligence massively unpopular; and ruthless enough to only cite one another and call themselves the "fathers of deep learning" rather than actually citing the originators of these ideas.
- backpropaganda 8y ago
  "merely" People deserve credit for taking an idea and running with it. Without Hinton's efforts (in 1987, 2006, or 2012), deep learning would have never seen the explosive growth.
  - srean 8y ago
    But is PR what Turing award is for though ?
    - mostafab 8y ago
      PR is what awards are all about
      - backpropaganda 8y ago
        PR is what all science is about. Without communicating ideas, science is not useful. The trio above managed to communicate deep learning ideas to the wider vision and NLP community such as by collaborating with them to come up with benchmarks which would prove usefulness of deep learning, but also to industry and general public. Schmidhuber did very little on these fronts, and even his technical contributions while laudable do not compare with the awardee's.
  - obviuosly 8y ago
    > People deserve credit for taking an idea and running with it. Yes, as popularizers. > deep learning would have never seen the explosive growth This is extremely unlikely to be true. I think the explosive growth occured because neurons and intelligence were unpopular in the 2000s because of PC, deferring progress in this area to occur explosively later. The Hinton cartel were not the only people who were aware that AI research "has become silly" (in his own words).
  - vbuwivbiu 8y ago
    "would have never..." it's not possible to make such an assertion unless you have access to at least one copy of this universe that doesn't include him. Maybe in all the other universes the exact same thing happened except someone else did it. Maybe in some of those universes, without him and his popularizing of this technique, some other better technique flourished. Who knows? Stating "would never have" is nothing more than telling a story.

tassl 8y ago

Direct link with the pdf version: http://people.idsia.ch/~juergen/DeepLearning2July2014.pdf http://people.idsia.ch/~juergen/DeepLearning2July2014.pdf Unfortunately, a good PR is definitely important for prizes.

freyir 8y ago

I could imagine this announcement will drive Schmidhuber up the wall. He's been very outspoken about how credit in the field should be assigned. That said, I don't know how you could argue against Hinton's impact on the field, or LeCun and Bengio's achievements.

dkural 8y ago

Fascinating! Thank you for sharing. The issues around the assignment of credit is as old as science and the concept of discovery; reading "The Invention of Science: A New History of the Scientific Revolution" by David Wootton really opened my eyes into how this is actually a fundamental feature of our modern conception of scientific discovery (sounds counter-intuitive.. I know!). It's like a societal "bug".

hobo_mark 8y ago

Bloomberg podcast about Schmidhuber, which was also quite entertaining: https://m.soundcloud.com/bloomberg-business/deeper-minds-read-aloud https://m.soundcloud.com/bloomberg-business/deeper-minds-rea...