6 ms·

List of "famous" ML people not to waste time on: - Gary Marcus - Juergen Schmidhuber - Pedro Domingos - Max Tegmark - Eliezer Yudkowsky Some context for p

List of "famous" ML people not to waste time on: - Gary Marcus - Juergen Schmidhuber - Pedro Domingos - Max Tegmark - Eliezer Yudkowsky Some context for people unfamiliar with ML research: the author, Schmidhuber, is well known for claiming that he should get credit for many ML ideas. Most ML researchers think that: - He doesn't deserve the credit he claims, in most if not all cases. - There's a few cases where his papers should have been cited and weren't. That's fairly common. - People do not get much credit for formulating an abstract idea in a paper or implementing it on a toy problem. Credit belongs to whoever actually makes it work. - Credit assignment in ML is not perfect but roughly works.

rcbdev 3y ago
While I generally agree with you in these specific cases, I feel like this is a shortsighted form of argument nonetheless. Coming up with an idea begets a scalable implementation of it and it's not very wise to ascribe absolute value solely on one side of the equation.
- TimPC 3y ago
  I think it's less about scalability and more about identifying specific choices that are promising. A lot of Schmidthuber's work paints out broad ideas in grand strokes and suggests hundreds of potential neural networks without evaluating what choices in that massive space are good. He then claims credit when other people identify the specific one or two of those hundred models (often with minor variations that make it not immediately obvious whether it perfectly fits the broad definition or not) that are actually promising.
  - calf 3y ago
    But if other people identify and further develop one or two out of his models, he should still be cited. If they did not use his work at all, coming ith one or two models independently, then that's a different situation. It's a bit of an honor code thing as well, it's hard to prove if somebody has a read a paper or not. But then there's a more stringent standard where one cites as part of surveying preexisting work, which can result in a vast list.
    - oldesthacker 3y ago
      Let's take an example: his "unnormalised linear Transformer," a neural network with "self-attention" published in 1992 under another name. It wasn't just an idea, it was implemented and tested in experiments. However, few people cared, because the computational hardware was so slow, and such networks were not yet practical. Decades later, the basic concepts were "reinvented" and renamed, and today they are really useful on much faster computers. Unquestionably, however, their origins must be cited by those who are applying them. Why are some people here even debating the generally recognised rules of scientific publishing mentioned in the paper: > The deontology of science requires: If one "reinvents" something that was already known, and only becomes aware of it later, one must at least clarify it later, and correctly give credit in all follow-up papers and presentations.
mahe_ 3y ago
> Credit belongs to whoever actually makes it work. This is just plain wrong. No working version without the idea.
- ben_w 3y ago
  Ideas are dime-a-dozen. I independently arrived at[0] something close to Max Tegmark's idea of the Mathematical Universe, almost nobody noticed and fewer still cared because I published it as a LiveJournal blog post whereas he fleshed it out into a whole book. I didn't get credit because I didn't do the hard work that deserves credit, I had the flash of inspiration and stopped after a few paragraphs of mediocre student philosophy. [0] and possibly predated, but I lost track of the date format when shifting from LJ to WP: https://kitsunesoftware.wordpress.com/2018/08/26/mathematical-universe-v-boltzmann-brains/ https://kitsunesoftware.wordpress.com/2018/08/26/mathematica...
GlenTheMachine 3y ago
Why Tegmark?
- erostrate 3y ago
  He talks a lot about the singularity but the biggest singularity I can see is in his ratio of "talks about AI" divided by "actual AI contributions".
  - senderista 3y ago
    It's not his field, what do you expect?
  - GlenTheMachine 3y ago
    There's nothing wrong with being a communicator to the public.
    - erostrate 3y ago
      Agreed, although I personally prefer when actual experts are communicating to the public. But that's not all he's doing, Tegmark also intends to influence global policy on AI research, random example: https://www.theguardian.com/technology/2023/sep/21/ai-focused-tech-firms-locked-race-bottom-warns-mit-professor-max-tegmark https://www.theguardian.com/technology/2023/sep/21/ai-focuse...
YeGoblynQueenne 3y ago
>> Credit belongs to whoever actually makes it work. That is according to whom? Is it a rule you just came up with or accepted practice? And if it's accepted practice, in what community is it accepted practice? Because where I publish and review there's really no such rule and credit belongs to the people who deserve credit for the work they've done that was useful to others.
- lowbloodsugar 3y ago
  Basically ideas are a dime a dozen. Sure, your idea might be a good one, but how do we spot your grain of sand is special when it looks the same as the rest of the desert? Essentially having an idea isn't useful to others. Demonstrating that your idea has legs is useful to others. I don't have to deal with citing papers, but I once had to deal with people pitching me ideas, wanting me to sign an NDA, in exchange for 50% of the revenue after I did all the actual work. Just out of curiosity, I signed one once. It was a fart app, IIRC. They thought a fart app needed an NDA, and that I'd then go do all the work and give them 50% because they "had the idea". It was so laughably sad. If you think these ideas are valuable, I have a beautiful clock for you. It is right twice a day. You'll have the same problem: you won't know when it's right. You'll need someone else's work to tell that.
  - shrimpx 3y ago
    > Basically ideas are a dime a dozen. There's a spectrum of ideas, from groundbreaking to "dime a dozen". In tech startups, and in almost all of computer science, most ideas are a dime a dozen, and the value is in the execution. But clearly, some ideas are groundbreaking. Einstein rightfully gets the credit for an on-paper hypothesis that wasn't proved until decades later via a chain of critical discoveries and experimental innovations by other people. It's legit to call it Einstin's relativity, and not Mossbauer/Hay's relativity.
    - throwaway81523 3y ago
      Ideas are a dime a dozen in the sense that the same idea will often occur to dozens of people, on a dime's worth of effort. Relativity theory wasn't anything like that. Einstein made predictions that no one else was making. When one of them from GR was confirmed a few years ago, Lenny Susskind famously marveled at the foresight, saying "they didn't call him Einstein for nothing!".
screye 3y ago
The sad part is that Schmidhuber is not a grifter by any means. If the turning prize for deep learning could go to 5 people instead of 3, he would very likely be on that list. His lab is excellent and was easily Europe's best deep learning lab for decades before it blew up. Some of his complaints are valid too. European labs often get ignored, and he has been sidelined despite being one of the most important people in deep learning himself. But man doesn't know when an argument runs out of gas. His claims get grander with every passing year. He would've just been the 'get off my lawn' grandpa of deep learning, but he somehow comes across as even more insufferable than that. I wonder if 2023 schmidhuber was created because the polite one from a decade ago was ignored. A sort of evil phase, if you will. I feel bad for him. He did get passed over of some deserved awards and recognition. But he reeks of resentment and thats never a good look.
- throwup238 3y ago
  > I feel bad for him. He did get passed over of some deserved awards and recognition. But he reeks of resentment and thats never a good look. It's a terrible look. We're in the middle of one of the biggest gold rushes in tech history and he's wasting time complaining when he claims to be one of its pioneers? That effort is much better invested in building stuff but I suspect he's fallen into the classic PI trap of writing grants all the time and leaving the real work to the rest of the faculty, atrophying his skills too much to do anything now that the industry is moving so quikcly.
  - logicchains 3y ago
    >atrophying his skills too much to do anything now that the industry is moving so quikcly. His recent papers are still cutting edge. He's already solved self-improving AI: https://arxiv.org/abs/2202.05780 https://arxiv.org/abs/2202.05780 , it just needs to be scaled up.
    - KRAKRISMOTT 3y ago
      His students you mean, otherwise he would be first author. Academia is just like capitalism, with credits substituting for capital. The capital owner (laboratory head/professor) always gets a cut of everything published.
logicchains 3y ago
>Most ML researchers think that: He doesn't deserve the credit he claims, in most if not all cases. That deserves a source. Especially for "all cases"; I don't think anyone who understands machine learning could read some of his earlier papers and still think Ian Goodfellow invented GANs.
- refulgentis 3y ago
  Frankly, I do, and it comes across as quibbling about categories and trying to define unnecessarily general taxonomic categories, as a reaction to positive reactions to other people. ex. in the article: "Goodfellow eventually admitted that my PM is adversarial...but emphasized that it's not generative. However, [it] is both adversarial and generative (its generator contains probabilistic units)...It is actually a generalized version of GANs." When you're at "actually, probabilities means generative, and actually you know what, even my initial claim was too specific: turns out its a generalized version of GANs", all in service of arguing a paper should have been cited in another paper, years after the other paper has been published, there's not much room for sympathy.
  - P-NP 3y ago
    This sounds a bit like a justification of plagiarism. In science, you must cite the original work.
- erostrate 3y ago
  Agreed, Ian Goodfellow wasn't the first to come up with the idea of jointly training a generator and discriminator. But he was the first to make it work for image generation with modern neural networks. For that Ian Goodfellow deserved to get most of the credit, and he did.
P-NP 3y ago
What a troll thing to write. User erostrate taking down the often so-called "father of modern AI" whose work is on erostrate's smartphone :-)
- erostrate 3y ago
  Are you saying Schmidhuber should also get credit for smartphones?

rcbdev 3y ago

While I generally agree with you in these specific cases, I feel like this is a shortsighted form of argument nonetheless. Coming up with an idea begets a scalable implementation of it and it's not very wise to ascribe absolute value solely on one side of the equation.

TimPC 3y ago
I think it's less about scalability and more about identifying specific choices that are promising. A lot of Schmidthuber's work paints out broad ideas in grand strokes and suggests hundreds of potential neural networks without evaluating what choices in that massive space are good. He then claims credit when other people identify the specific one or two of those hundred models (often with minor variations that make it not immediately obvious whether it perfectly fits the broad definition or not) that are actually promising.
- calf 3y ago
  But if other people identify and further develop one or two out of his models, he should still be cited. If they did not use his work at all, coming ith one or two models independently, then that's a different situation. It's a bit of an honor code thing as well, it's hard to prove if somebody has a read a paper or not. But then there's a more stringent standard where one cites as part of surveying preexisting work, which can result in a vast list.
  - oldesthacker 3y ago
    Let's take an example: his "unnormalised linear Transformer," a neural network with "self-attention" published in 1992 under another name. It wasn't just an idea, it was implemented and tested in experiments. However, few people cared, because the computational hardware was so slow, and such networks were not yet practical. Decades later, the basic concepts were "reinvented" and renamed, and today they are really useful on much faster computers. Unquestionably, however, their origins must be cited by those who are applying them. Why are some people here even debating the generally recognised rules of scientific publishing mentioned in the paper: > The deontology of science requires: If one "reinvents" something that was already known, and only becomes aware of it later, one must at least clarify it later, and correctly give credit in all follow-up papers and presentations.

mahe_ 3y ago

> Credit belongs to whoever actually makes it work. This is just plain wrong. No working version without the idea.

ben_w 3y ago
Ideas are dime-a-dozen. I independently arrived at[0] something close to Max Tegmark's idea of the Mathematical Universe, almost nobody noticed and fewer still cared because I published it as a LiveJournal blog post whereas he fleshed it out into a whole book. I didn't get credit because I didn't do the hard work that deserves credit, I had the flash of inspiration and stopped after a few paragraphs of mediocre student philosophy. [0] and possibly predated, but I lost track of the date format when shifting from LJ to WP: https://kitsunesoftware.wordpress.com/2018/08/26/mathematical-universe-v-boltzmann-brains/ https://kitsunesoftware.wordpress.com/2018/08/26/mathematica...

GlenTheMachine 3y ago

Why Tegmark?

erostrate 3y ago
He talks a lot about the singularity but the biggest singularity I can see is in his ratio of "talks about AI" divided by "actual AI contributions".
- senderista 3y ago
  It's not his field, what do you expect?
- GlenTheMachine 3y ago
  There's nothing wrong with being a communicator to the public.
  - erostrate 3y ago
    Agreed, although I personally prefer when actual experts are communicating to the public. But that's not all he's doing, Tegmark also intends to influence global policy on AI research, random example: https://www.theguardian.com/technology/2023/sep/21/ai-focused-tech-firms-locked-race-bottom-warns-mit-professor-max-tegmark https://www.theguardian.com/technology/2023/sep/21/ai-focuse...

YeGoblynQueenne 3y ago

>> Credit belongs to whoever actually makes it work. That is according to whom? Is it a rule you just came up with or accepted practice? And if it's accepted practice, in what community is it accepted practice? Because where I publish and review there's really no such rule and credit belongs to the people who deserve credit for the work they've done that was useful to others.

lowbloodsugar 3y ago
Basically ideas are a dime a dozen. Sure, your idea might be a good one, but how do we spot your grain of sand is special when it looks the same as the rest of the desert? Essentially having an idea isn't useful to others. Demonstrating that your idea has legs is useful to others. I don't have to deal with citing papers, but I once had to deal with people pitching me ideas, wanting me to sign an NDA, in exchange for 50% of the revenue after I did all the actual work. Just out of curiosity, I signed one once. It was a fart app, IIRC. They thought a fart app needed an NDA, and that I'd then go do all the work and give them 50% because they "had the idea". It was so laughably sad. If you think these ideas are valuable, I have a beautiful clock for you. It is right twice a day. You'll have the same problem: you won't know when it's right. You'll need someone else's work to tell that.
- shrimpx 3y ago
  > Basically ideas are a dime a dozen. There's a spectrum of ideas, from groundbreaking to "dime a dozen". In tech startups, and in almost all of computer science, most ideas are a dime a dozen, and the value is in the execution. But clearly, some ideas are groundbreaking. Einstein rightfully gets the credit for an on-paper hypothesis that wasn't proved until decades later via a chain of critical discoveries and experimental innovations by other people. It's legit to call it Einstin's relativity, and not Mossbauer/Hay's relativity.
  - throwaway81523 3y ago
    Ideas are a dime a dozen in the sense that the same idea will often occur to dozens of people, on a dime's worth of effort. Relativity theory wasn't anything like that. Einstein made predictions that no one else was making. When one of them from GR was confirmed a few years ago, Lenny Susskind famously marveled at the foresight, saying "they didn't call him Einstein for nothing!".

screye 3y ago

The sad part is that Schmidhuber is not a grifter by any means. If the turning prize for deep learning could go to 5 people instead of 3, he would very likely be on that list. His lab is excellent and was easily Europe's best deep learning lab for decades before it blew up. Some of his complaints are valid too. European labs often get ignored, and he has been sidelined despite being one of the most important people in deep learning himself. But man doesn't know when an argument runs out of gas. His claims get grander with every passing year. He would've just been the 'get off my lawn' grandpa of deep learning, but he somehow comes across as even more insufferable than that. I wonder if 2023 schmidhuber was created because the polite one from a decade ago was ignored. A sort of evil phase, if you will. I feel bad for him. He did get passed over of some deserved awards and recognition. But he reeks of resentment and thats never a good look.

throwup238 3y ago
> I feel bad for him. He did get passed over of some deserved awards and recognition. But he reeks of resentment and thats never a good look. It's a terrible look. We're in the middle of one of the biggest gold rushes in tech history and he's wasting time complaining when he claims to be one of its pioneers? That effort is much better invested in building stuff but I suspect he's fallen into the classic PI trap of writing grants all the time and leaving the real work to the rest of the faculty, atrophying his skills too much to do anything now that the industry is moving so quikcly.
- logicchains 3y ago
  >atrophying his skills too much to do anything now that the industry is moving so quikcly. His recent papers are still cutting edge. He's already solved self-improving AI: https://arxiv.org/abs/2202.05780 https://arxiv.org/abs/2202.05780 , it just needs to be scaled up.
  - KRAKRISMOTT 3y ago
    His students you mean, otherwise he would be first author. Academia is just like capitalism, with credits substituting for capital. The capital owner (laboratory head/professor) always gets a cut of everything published.

logicchains 3y ago

>Most ML researchers think that: He doesn't deserve the credit he claims, in most if not all cases. That deserves a source. Especially for "all cases"; I don't think anyone who understands machine learning could read some of his earlier papers and still think Ian Goodfellow invented GANs.

refulgentis 3y ago
Frankly, I do, and it comes across as quibbling about categories and trying to define unnecessarily general taxonomic categories, as a reaction to positive reactions to other people. ex. in the article: "Goodfellow eventually admitted that my PM is adversarial...but emphasized that it's not generative. However, [it] is both adversarial and generative (its generator contains probabilistic units)...It is actually a generalized version of GANs." When you're at "actually, probabilities means generative, and actually you know what, even my initial claim was too specific: turns out its a generalized version of GANs", all in service of arguing a paper should have been cited in another paper, years after the other paper has been published, there's not much room for sympathy.
- P-NP 3y ago
  This sounds a bit like a justification of plagiarism. In science, you must cite the original work.
erostrate 3y ago
Agreed, Ian Goodfellow wasn't the first to come up with the idea of jointly training a generator and discriminator. But he was the first to make it work for image generation with modern neural networks. For that Ian Goodfellow deserved to get most of the credit, and he did.

P-NP 3y ago

What a troll thing to write. User erostrate taking down the often so-called "father of modern AI" whose work is on erostrate's smartphone :-)

erostrate 3y ago
Are you saying Schmidhuber should also get credit for smartphones?