9 ms·
Turing awardees republished key methods and ideas without credit
- Imnimo 3y agoJust from the title I knew exactly which awardees we were talking about, and who was going to be doing the talking.
- meowface 3y agoI guessed it was them but only because they're the only 3 I know off-hand.
- hiddencost 3y agoSchmidhuber really needs to stop. He's been beating this drum for decades and he's wrong.
- martopix 3y agoIf a guy becomes famous almost exclusively for having a beef with someone else (and this someone else really doesn't need to beat any drum in order to be famous), then something is wrong. Basically these days this is what Schmidhuber is associated to. Which is kind of sad, really.
- logicchains 3y ago>If a guy becomes famous almost exclusively for having a beef with someone else He was famous well before that for having one of the best labs in Europe, and many key papers in early deep learning. He's only famous for the beef among people who aren't familiar with his earlier work and the huge contributions it made to the field.
- caddemon 3y agoIt's hard not to think of the beef when you think of him now though, even if you already knew some of his work. To be clear I don't think that should take away from what he did accomplish, but it's like a Pavlovian association at this point to also have his behavior come to mind.
- martopix 3y agoI should not have said "becomes famous for...". I meant that today, his name is mostly associated with that, when people talk about him.
- logicchains 3y agoSo he should just give up and let people get away with plagiarism? Do you teach your kid to just give in and let bullies get their way?
- gexaha 3y agoI wonder what Schmidhuber colleagues think of all of this.
- gillesjacobs 3y agoWithin the field his name has become a punchline to joke about self-aggrandizing. But yeah, you honestly have to wonder what it is like day-to-day working with someone with this level of delusions of grandeur.
- logicchains 3y ago>But yeah, you honestly have to wonder what it is like day-to-day working with someone with this level of delusions of grandeur. It's not delusions of grandeur; anyone with half a brain who read his early papers would see he clearly came up with the idea of a GAN well before Ian Goodfellow.
- peterfirefly 3y agoThat particular claim seems to be true, yeah.
- gexaha 3y agoI mean, his co-authors, or colleagues at IDSIA
- RcouF1uZ4gsC 3y agoReading the article and some of the links make me feel like the author Jürgen Schmidhuber is the academic version of the patent troll. It sounds like he published some theoretical musings back in 1990s without any real practical implementation that did anything useful and since then has run around accusing AI researchers who actually produced concrete research and techniques to get are actually in use today of plagiarism.
- aborsy 3y agoRegardless, it should be easy to verify. He cites a paper and says it’s done in this earlier paper. On the other hand, if his claims are verified, some of these research “discoveries” are simply common sense and would occur to most people working on the subject. HLB were awarded to a good extent because they worked on deep learning at the right time. Deep learning became hugely practical, and outperformed the state of the art in many applications. HL also worked for major companies.
- logicchains 3y ago>It sounds like he published some theoretical musings back in 1990s without any real practical implementation He published working models back then, the problem was compute power was very limited. In the past decade deep learning took off, people took his models, renamed then and ran them on vastly more powerful computers, to great success, then failed to cite him.
- vbarrielle 3y agoIt's actually a common problem in lots of computing related research communities. Papers older than 10 years ago, sometimes even 5 years ago, are ignored. Because the results do not compare in terms of computing power, and because reimplementing old papers is often very hard because they are often too vague.
- P-NP 3y ago> without any real practical implementation Don't you know that billions of people are using his work on a daily basis on their smartphone? CV: https://people.idsia.ch/~juergen/cv.html https://people.idsia.ch/~juergen/cv.html
- harveywi 3y agoLeCuna (noun): An empty space in a list of citations where the works of Jürgen Schmidhuber should appear.
- SeanLuke 3y agoWe published a paper a while back in a genetic programming conference, and before the paper had been published (only the title was announced) Schmidhuber guessed that we had not cited his work. He very publically put us through the wringer for not citing him as the inventor of the concept were were examining. In fact we had not cited him: but among our dozen or so previous examples, we had cited two others which long predated his work and were the actual seminal papers. He had failed to cite them himself.
- tines 3y ago> He very publically put us through the wringer for not citing him as the inventor of the concept were were examining He put you through the wringer for not citing him in the announcement of the title?
- calf 3y agoBut it is possible that he ought to have been cited as well? It doesn't matter if he's a hypocrite. The justification ultimately has to be, "Our paper did not use your work in any substantial way therefore a citation is not needed." I can see how citations can get very complex for the purposes of scientific credit, traditionally there's a self-policed aspect to it in every academic community; however, in this century as advanced research gets more complex and globalized, it might be time for a more rigorous and neutral process of doing so.
- lacker 3y agoWhenever you cite papers just because you're "politically supposed to", even though you don't think it's really intellectually useful for anyone to be aware of the cited paper, you're making the citations less useful for everybody. Schmidhuber is only the worst of many abusers of the citation system. Everyone knows that anonymous reviewers are more likely to approve of people who cite their work. Someone is politically powerful? Better cite their paper, even though you don't really think it's a good paper and you didn't use it. Schmidhuber is cited far more often than he should be, because people just don't want to deal with this crap, and the cost of sticking in one more citation is very low.
- pavel_lishin 3y agoCue the next HBomberguy video, please.
- erostrate 3y agoList of "famous" ML people not to waste time on: - Gary Marcus - Juergen Schmidhuber - Pedro Domingos - Max Tegmark - Eliezer Yudkowsky Some context for people unfamiliar with ML research: the author, Schmidhuber, is well known for claiming that he should get credit for many ML ideas. Most ML researchers think that: - He doesn't deserve the credit he claims, in most if not all cases. - There's a few cases where his papers should have been cited and weren't. That's fairly common. - People do not get much credit for formulating an abstract idea in a paper or implementing it on a toy problem. Credit belongs to whoever actually makes it work. - Credit assignment in ML is not perfect but roughly works.
- rcbdev 3y agoWhile I generally agree with you in these specific cases, I feel like this is a shortsighted form of argument nonetheless. Coming up with an idea begets a scalable implementation of it and it's not very wise to ascribe absolute value solely on one side of the equation.
- TimPC 3y agoI think it's less about scalability and more about identifying specific choices that are promising. A lot of Schmidthuber's work paints out broad ideas in grand strokes and suggests hundreds of potential neural networks without evaluating what choices in that massive space are good. He then claims credit when other people identify the specific one or two of those hundred models (often with minor variations that make it not immediately obvious whether it perfectly fits the broad definition or not) that are actually promising.
- calf 3y agoBut if other people identify and further develop one or two out of his models, he should still be cited. If they did not use his work at all, coming ith one or two models independently, then that's a different situation. It's a bit of an honor code thing as well, it's hard to prove if somebody has a read a paper or not. But then there's a more stringent standard where one cites as part of surveying preexisting work, which can result in a vast list.
- 1024core 3y agoLife would be so boring without Schmidhuber.
- gedy 3y agoAs an outsider to this space, it seems suspect that all of the people he has a beef with (like LaCun) will gladly cite others and predecessors, but not Schmidhuber. Has he explained his reasons for thinking there is some conspiracy? Otherwise it reflects badly on his assessment of himself, or possibly his mental state.
- daveguy 3y agoIt's nice of Schmidhuber to point out the quality papers with theoretical advancements and actual validations that fix the intellectual problems with his weak musings.
- YeGoblynQueenne 3y agoThis whole comment section is full of absolutely unacceptable ad-hominem attacks on Schmidhuber, from people who most likely haven't even read any of the works in question and are certainly not showing any of the "intellectual curiosity" this site is supposed to be about. Anyone who cares about academic integrity should at least not attack someone complaining of plagiarism. That sort of attack is the academic equivalent of blaming the victim. If even half of Schmidhuber's accusations have a basis that's still a major academic scandal of epic proportions.
- peterfirefly 3y agoI tried reading one of his papers that he claimed contained an important concept avant la lettre. It was unreadable and it required an almost Marxist torture of the text to make it confess (weakly and unconvincingly) that it contained anything remotely like the idea he claimed it did.
- mavelikara 3y agoWhat is "Marxist torture"?
- logicchains 3y ago>It was unreadable Which paper was that; I've always found his writing relatively clear. Are you sure it wasn't just the case that you didn't understand the paper?
- matkoniecz 3y ago> Anyone who cares about academic integrity should at least not attack someone complaining of plagiarism What about cases when accusation was unreasonable and not matching reality?
- dr_dshiv 3y agoI came to Schmidhuber's work learning about his approach on creativity and curiosity and I loved it so much. It’s still so relevant. And beautiful, actually. https://arxiv.org/pdf/0709.0674.pdf https://arxiv.org/pdf/0709.0674.pdf
- qzw 3y agoShouldn't the title be "Turing Awardees Accused of Republishing..." rather than how it currently reads?
- yorwba 3y agoNo, because the article author is the one making the accusations.
- deleted 3y ago[deleted]
- 33a 3y agoSomehow I knew it was Schmidhuber before I clicked the link.
- renecito 3y agoShocking! Successful people getting credited for someone else work.
- cs702 3y agoCopying and pasting Urban Dictionary's definitions of "to schmidhuber"[a], "schmidhuber"[b] and "schmidhubered"[c]: --- to schmidhuber When you publicly claim that someone else's idea that is remotely resembling your own is stolen from you. --- schmidhuber 1) To interject for a moment and explain how one's recent popular idea is a few transformations away from your 1991 paper. 2) To miraculously produce fifty years of relevant literature after someone claims to trace the origin of an idea in a particular work. --- schmidhubered Being "schmidhubered" looks something like this: 1) Invent something brilliant that no one cares about. Experience derision. 2) That thing becomes popular years later. Someone else is given credit for inventing it. That person appears in the New York Times and is declared smartest person alive. 3) Go on a campaign explaining the situation and how you are the rightful inventor and thus the rightful Smartest Person Alive. 4) Everyone accuses you of being a sore loser and no one takes you seriously. 5) A verb is named after you. --- [a] https://www.urbandictionary.com/define.php?term=to+schmidhuber https://www.urbandictionary.com/define.php?term=to+schmidhub... [b] https://www.urbandictionary.com/define.php?term=schmidhuber https://www.urbandictionary.com/define.php?term=schmidhuber [c] https://www.urbandictionary.com/define.php?term=schmidhubered https://www.urbandictionary.com/define.php?term=schmidhubere...
- deleted 3y ago[deleted]
- P-NP 3y agoI can see some angry comments here, but so far I have not seen any facts that refute his claims. Once I spent a long time reviewing a related paper on Hacker News, and I think he is right about disputes B1, B2, B5, H2, H4, H5. I'd have to study the others more closely: B: Priority disputes with Dr. Bengio (original date v Bengio's date): B1: Generative adversarial networks or GANs (1990 v 2014) B2: Vanishing gradient problem (1991 v 1994) B3: Metalearning (1987 v 1991) B4: Learning soft attention (1991-93 v 2014) for Transformers etc. B5: Gated recurrent units (2000 v 2014) B6: Auto-regressive neural nets for density estimation (1995 v 1999) B7: Time scale hierarchy in neural nets (1991 v 1995) H: Priority disputes with Dr. Hinton (original date v Hinton's date): H1: Unsupervised/self-supervised pre-training for deep learning (1991 v 2006) H2: Distilling one neural net into another neural net (1991 v 2015) H3: Learning sequential attention with neural nets (1990 v 2010) H4: NNs program NNs: fast weight programmers (1991 v 2016) and linear Transformers H5: Speech recognition through deep learning (2007 v 2012) H6: Biologically plausible forward-only deep learning (1989, 1990, 2021 v 2022) L: Priority disputes with Dr. LeCun (original date v LeCun's date): L1: Differentiable architectures / intrinsic motivation (1990 v 2022) L2: Multiple levels of abstraction and time scales (1990-91 v 2022) L3: Informative yet predictable representations (1997 v 2022) L4: Learning to act largely by observation (2015 v 2022)
- P-NP 3y agoIn the past few hours I have had more time to look at the entire piece and download some of the referenced papers. So far I haven't found any claim that's factually inaccurate. I think there is a reason why the ACM Turing awardees have never tried to defend themselves by presenting facts to the contrary: because they can't. This might get interesting: > The "Policy for Honors Conferred by ACM"[ACM23] mentions that ACM "retains the right to revoke an Honor previously granted if ACM determines that it is in the best interests of the field to do so." So I ask ACM to evaluate the presented evidence and decide about further actions.
- senderista 3y agoThis guy has been the biggest blowhard in AI for years if not decades.
- oldesthacker 3y agoThe machine learning field as a whole has a huge credit assignment problem. This post seems to encourage other ML researchers to come out with their own priority disputes. Tomas Mikolov just aired his grievances: > I wanted to popularize neural language models by improving Google Translate. I did start collaboration with Franz Och and his team, during which time I proposed a couple of models that could either complement the phrase-based machine translation, or even replace it. I came up (actually even before joining Google) with a really simple idea to do end-to-end translation by training a neural language model on pairs of sentences (say French - English), and then use the generation mode to produce translation after seeing the first sentence. It worked great on short sentences, but not so much on the longer ones. I discussed this project many times with others in Google Brain - mainly Quoc and Ilya - who took over this project after I moved to Facebook AI. I was quite negatively surprised when they ended up publishing my idea under now famous name "sequence to sequence" where not only I was not mentioned as a co-author, but in fact my former friends forgot to mention me also in the long Acknowledgement section, where they thanked personally pretty much every single person in Google Brain except me. This was the time when money started flowing massively into AI and every idea was worth gold. It was sad to see the deep learning community quickly turn into some sort of Game of Thrones. Money and power certainly corrupts people... Reddit post: "Tomas Mikolov is the true father of sequence-to-sequence" https://www.reddit.com/r/MachineLearning/comments/18jzxpf/d_tomas_mikolov_is_the_true_father_of/ https://www.reddit.com/r/MachineLearning/comments/18jzxpf/d_...