3 ms·

Every so often Schmidhuber is brought back to the front-page of HN, people will argue that he "invented it all" while others will say that he's a-posteriori cla

by belval 1y ago

Every so often Schmidhuber is brought back to the front-page of HN, people will argue that he "invented it all" while others will say that he's a-posteriori claiming all the good ideas were his. Relativity Priority Dispute: https://en.wikipedia.org/wiki/Relativity_priority_dispute https://en.wikipedia.org/wiki/Relativity_priority_dispute We all stand on the shoulders of giants, things can be invented and reinvented and ideas can appear twice in a vacuum.

kleiba 1y ago
But as far as I understand, Schmidhuber's claim is more severe: namely that Bengio, Hinton and LeCun intentionally failed to cite prior work by others (including himself) but instead only cited each other in order to boost their respective scientific reputation. I personally think that he's not doing himself or his argument of favor by presenting it the way he does. While he basically argues that science should be totally objective and neutral, there's no denying that if you put yourself in a less likeable light, you're not going to make any friends. On the other hand, he's gone at length with compiling detailed references to support his points. I can appreciate that because it makes his argument a lot less hand-wavey: you can go to his blog and compare the cited references yourself. Except that I couldn't because I'm not an ML expert.
- jll29 1y ago
  I have seen many cases where people -- accidentally as well as intentionally -- copied or re-invented the work of others (a friend posted on LinkedIn that someone else plagiarized his whole Ph.D. thesis, including the title, so only the name was changed; only the references to seperately published papers on which the individual chapters were based at the end still had my friend's name in it, so you could see it was a fake thesis). If a bona fide scientist makes a mistake about missing attribution, they would correct it as soon as possible. Many, however, would not correct such a re-discovery, because it's embarrassing. But the worst is when people don't even imagine anything like what they are working on could already exist, and they don't even bother finding and reading related work -- in other words, ignorance. Science deserves better, but there are more and more ignorant folks around that want to ignore all work before them.
  - godelski 1y ago
    > Many, however, would not correct such a re-discovery, because it's embarrassing. This is a culture thing that needs to change. I'm a pretty big advocate of open publishing and avoiding the "review process" as it stands today. The reason is because we shouldn't be chasing these notions of novelty and "impact". They are inherently subjective and lead to these issues of credit. Your work isn't diminished because you independently invented it, rather, that strengthens your work. There's more evidence! Everything is incremental and so all this stuff does is makes us focus more on trying to show our uniqueness rather than showing our work. The point of publishing is to communicate. The peer review process only happens post communicating: when people review, replicate, build on, or build against. We're just creating an overly competitive environment. It is only "embarrassing" because it "undermines" the work. It only "undermines" the work because how we view credit. Consider this as a clear example. Suppose you want to revisit a work but just scale it up and use on modern hardware. You could get state of the art results but if you admit to such a thing with no claimed changes (let's say you literally just increase number of layers) you'll never get published. You'll get responses about how we "already knew this" and "obviously it scales". But no one tested it... right? That's just bad for science. It's bad if we can't do mundane boring shit.
- srean 1y ago
  > Hinton and LeCun intentionally failed to cite prior work by others (including himself) This is a well established and easy to verify fact. A peer of mine from Hinton's lab once told a story of how one researcher (presumably from his lab or from one of his coworker's lab) intentionally mispelled citations so that the author's citation count does not go up on google-scholar, citeseer etc. He did not name the names though.
Lerc 1y ago
I can see how someone could feel like that if they looked at the world in a particular way. I have had plenty of ideas in the last few years that I have played with that I have seen published in papers in the following months. Rather than feeling like "I did it first" I feel gratified that not only was I on the right track, but someone else has done the hard slog. Most papers are not published by people who had the idea the day before. Their work goes back further than that. Refining the idea, testing it and then presenting the results takes time, sometimes years, occasionally decades. If this happens to you, don't think "Hey! That idea belongs to me!". Thank them for proving you right. Now if they patent it, that's a different story. I don't think the ideas that sometimes float through my brain belong to me, but I'm not keen on them belonging to someone else either.
- kleiba 1y ago
  I think that's slightly mispresenting Schmidhuber's case though, because he does not just say "oh, I already had that same idea before you, I just follow up on it". He is usually referring to work that he or members of his group (or third-party researchers for that amatter) did, in fact, already publish.
- Kranar 1y ago
  This claim is sheer hubris. There is a big difference between spending years researching and working on a nascent subject and publishing it in academic journals, only to have someone else come along and use most of your ideas without attribution and reaping huge rewards for doing so... and having some random ideas float about in your head on a nice summer afternoon by the lake, and then a few months later find out that someone else also shared those same ideas and managed to work through them into something fruitful. Now whether what Schmidhuber claims is what actually happened or not I don't know... but that is his claim and it's fundamentally different from what you are describing.

by belval 1y ago

kleiba 1y ago

But as far as I understand, Schmidhuber's claim is more severe: namely that Bengio, Hinton and LeCun intentionally failed to cite prior work by others (including himself) but instead only cited each other in order to boost their respective scientific reputation. I personally think that he's not doing himself or his argument of favor by presenting it the way he does. While he basically argues that science should be totally objective and neutral, there's no denying that if you put yourself in a less likeable light, you're not going to make any friends. On the other hand, he's gone at length with compiling detailed references to support his points. I can appreciate that because it makes his argument a lot less hand-wavey: you can go to his blog and compare the cited references yourself. Except that I couldn't because I'm not an ML expert.

jll29 1y ago
I have seen many cases where people -- accidentally as well as intentionally -- copied or re-invented the work of others (a friend posted on LinkedIn that someone else plagiarized his whole Ph.D. thesis, including the title, so only the name was changed; only the references to seperately published papers on which the individual chapters were based at the end still had my friend's name in it, so you could see it was a fake thesis). If a bona fide scientist makes a mistake about missing attribution, they would correct it as soon as possible. Many, however, would not correct such a re-discovery, because it's embarrassing. But the worst is when people don't even imagine anything like what they are working on could already exist, and they don't even bother finding and reading related work -- in other words, ignorance. Science deserves better, but there are more and more ignorant folks around that want to ignore all work before them.
- godelski 1y ago
  > Many, however, would not correct such a re-discovery, because it's embarrassing. This is a culture thing that needs to change. I'm a pretty big advocate of open publishing and avoiding the "review process" as it stands today. The reason is because we shouldn't be chasing these notions of novelty and "impact". They are inherently subjective and lead to these issues of credit. Your work isn't diminished because you independently invented it, rather, that strengthens your work. There's more evidence! Everything is incremental and so all this stuff does is makes us focus more on trying to show our uniqueness rather than showing our work. The point of publishing is to communicate. The peer review process only happens post communicating: when people review, replicate, build on, or build against. We're just creating an overly competitive environment. It is only "embarrassing" because it "undermines" the work. It only "undermines" the work because how we view credit. Consider this as a clear example. Suppose you want to revisit a work but just scale it up and use on modern hardware. You could get state of the art results but if you admit to such a thing with no claimed changes (let's say you literally just increase number of layers) you'll never get published. You'll get responses about how we "already knew this" and "obviously it scales". But no one tested it... right? That's just bad for science. It's bad if we can't do mundane boring shit.
srean 1y ago
> Hinton and LeCun intentionally failed to cite prior work by others (including himself) This is a well established and easy to verify fact. A peer of mine from Hinton's lab once told a story of how one researcher (presumably from his lab or from one of his coworker's lab) intentionally mispelled citations so that the author's citation count does not go up on google-scholar, citeseer etc. He did not name the names though.

Lerc 1y ago

I can see how someone could feel like that if they looked at the world in a particular way. I have had plenty of ideas in the last few years that I have played with that I have seen published in papers in the following months. Rather than feeling like "I did it first" I feel gratified that not only was I on the right track, but someone else has done the hard slog. Most papers are not published by people who had the idea the day before. Their work goes back further than that. Refining the idea, testing it and then presenting the results takes time, sometimes years, occasionally decades. If this happens to you, don't think "Hey! That idea belongs to me!". Thank them for proving you right. Now if they patent it, that's a different story. I don't think the ideas that sometimes float through my brain belong to me, but I'm not keen on them belonging to someone else either.

kleiba 1y ago
I think that's slightly mispresenting Schmidhuber's case though, because he does not just say "oh, I already had that same idea before you, I just follow up on it". He is usually referring to work that he or members of his group (or third-party researchers for that amatter) did, in fact, already publish.
Kranar 1y ago
This claim is sheer hubris. There is a big difference between spending years researching and working on a nascent subject and publishing it in academic journals, only to have someone else come along and use most of your ideas without attribution and reaping huge rewards for doing so... and having some random ideas float about in your head on a nice summer afternoon by the lake, and then a few months later find out that someone else also shared those same ideas and managed to work through them into something fruitful. Now whether what Schmidhuber claims is what actually happened or not I don't know... but that is his claim and it's fundamentally different from what you are describing.