7 ms·
I think Schmidhuber should be up there. LSTMs are everywhere, not just NLP. The latest Starcraft 2 bot from Deepmind uses LSTMs extensively in its architecture.
by li4ick 8y ago
I think Schmidhuber should be up there. LSTMs are everywhere, not just NLP. The latest Starcraft 2 bot from Deepmind uses LSTMs extensively in its architecture.
- high_derivative 8y agoYes seems political to snub him, but then ACM is an American association.
- theWheez 8y agoDr. LeCun is a French native, for what it's worth
- screye 8y agoBut has spent almost all of his professional career in A&T labs, NYU, FB and Toronto. That's not true with Schmidhuber.
- dirtybird04 8y agoumm two of the award recipients are Canadians
- deleted 8y ago[deleted]
- high_derivative 8y agoI encourage downvoters to read the article below for a perspective on how LeCun, Hinton and Bengio may have unfairly excluded a wealth of prior work in their version of deep learning history: http://people.idsia.ch/~juergen/deep-learning-conspiracy.html http://people.idsia.ch/~juergen/deep-learning-conspiracy.htm... Note for people put off by the link: The conspiracy refers to the title of a paper by the Turing award winners, not Schmidhuber's.
- nabla9 8y agoSchmidhuber's significance in the field is not at the same level. He invented one type of neural network, but the "Canadian mafia" launched the whole Deep Learning revolution with multiple breakthroughs and theoretical understanding. LSTM's are clever and they were bleeding edge up to 2014, but once they were understood better as a bypass mechanism, attention, context vectors and averaging networks and causal convolution are starting to replace them.
- Nimitz14 8y agoSchmidhuber did way more than LSTMs. And they still are SOTA on many tasks. You clearly are not very familiar with the field.
- deleted 8y ago[deleted]
- pault 8y agoThat third sentence is not necessary.
- nabla9 8y agoI'm not saying that Schmidhuber is not important pioneer in the field. When you put researchers or groups into order of importance, the work from this trio comes before others, including Schmidhuber. The Turing Award is given to those at the top and it's not inclusive. If I could give Turing Award for someone in the field who has not received it yet, I would give it to Vladimir Vapnik.
- Nimitz14 8y agoSorry.
- gambler 8y ago>If I could give Turing Award for someone in the field who has not received it yet, I would give it to Vladimir Vapnik. So would I, but I don't think that will happen. He's is pretty much an antithesis to everything Deep Learning is about. Theory-first over trial-and-error, math over intuition, small datasets over big data, advances in understanding vs advances in results. I haven't seen a single discussion of his paper on combining classifiers[1] anywhere on the web. [1] - http://jmlr.csail.mit.edu/papers/volume17/16-137/16-137.pdf http://jmlr.csail.mit.edu/papers/volume17/16-137/16-137.pdf
- h8hawk 8y agoSchmidhuber didn't invent LSTM Just by himself. Original papaer was published by Sepp Hochreiter and Schmidhuber.
- jmmcd 8y agoYes, and the vast, vast majority of important papers by the three winners were co-authored in the same way with supervisors, graduate students or other collaborators.
- tanilama 8y agoIf LSTM is up there, maybe Transformer authors should also be listed somewhere? I mean LSTM is fine and all, IIRC, it only becomes popular after one of Hinton's students put it to work? And that is like in 2013. Similar tales to the CNN as well. Hinton/Lecun pioneered early BP research, which is significant and fundamental. Bengio has attention mechanism/GAN, and in early days greedy layer wise pretraining for RBM (which isn't something end up today, but pertaining is a thing back in pre 2010 era). Not saying LSTM isn't significant, but his achievements aren't quite there with those 3, looking closer.
- moultano 8y agoI think we need a few more years to see whether LSTMs retain their importance over time. Awarding it now seems premature.