7 ms·

Jürgen Schmidhuber：the Father of Generative AI Without Turing Award

115 points by kleiba 1y ago

jll29 1y ago
I highly recommend going through his report; it is a solid basis for the curious to verify for themselves who invented what and when. Intellectual honesty is very important. That includes not selling something as new if there are already publications of the idea; worse than to hide one's knowledge of prior art is to not even look in the first place ("ostrich strategy").
careful_ai 1y ago
This post nails the common pain point: raw GenAI outputs feel detached from real codebases and impose an overhead that often negates any speed gains from AI. The idea that generative coding tools are "offloading the easy stuff while leaving the hard, domain-specific thinking to humans" mirrors my own experience. There’s a strange cognitive tax that comes with trying to validate AI-generated code—especially when it’s syntactically right but semantically wrong in subtle ways. What I’ve found most frustrating is that these tools often lack a sense of context. Not just in terms of project architecture, but also in terms of intent and long-term maintainability. I’ve ended up spending more time reviewing and rewriting than if I had just tackled it manually. Still, I’m hopeful. When paired with lightweight human-in-the-loop workflows—like using AI to explain legacy logic or spot inconsistencies—it can be a decent thought partner. But as a coder’s autopilot? Not there yet.
nharada 1y ago
Doesn’t he know the Turing Award is really just a generalization of the Fields Medal, an award that actually came years earlier?
- logicchains 1y ago
  I'm sure he wouldn't object to a Fields Medal either.
- triceratops 1y ago
  I chuckled but I also maybe didn't understand. Is the joke that computer science is a generalization of math? That can't be rigth.
  - dgacmu 1y ago
    The joke is that schmidhuber is known for (rightly or wrongly) pointing to modern contributions in deep neural networks and saying they're just a trivial generalization/adaptation/etc. of work he did 30 years ago.
belval 1y ago
Every so often Schmidhuber is brought back to the front-page of HN, people will argue that he "invented it all" while others will say that he's a-posteriori claiming all the good ideas were his. Relativity Priority Dispute: https://en.wikipedia.org/wiki/Relativity_priority_dispute https://en.wikipedia.org/wiki/Relativity_priority_dispute We all stand on the shoulders of giants, things can be invented and reinvented and ideas can appear twice in a vacuum.
- kleiba 1y ago
  But as far as I understand, Schmidhuber's claim is more severe: namely that Bengio, Hinton and LeCun intentionally failed to cite prior work by others (including himself) but instead only cited each other in order to boost their respective scientific reputation. I personally think that he's not doing himself or his argument of favor by presenting it the way he does. While he basically argues that science should be totally objective and neutral, there's no denying that if you put yourself in a less likeable light, you're not going to make any friends. On the other hand, he's gone at length with compiling detailed references to support his points. I can appreciate that because it makes his argument a lot less hand-wavey: you can go to his blog and compare the cited references yourself. Except that I couldn't because I'm not an ML expert.
  - jll29 1y ago
    I have seen many cases where people -- accidentally as well as intentionally -- copied or re-invented the work of others (a friend posted on LinkedIn that someone else plagiarized his whole Ph.D. thesis, including the title, so only the name was changed; only the references to seperately published papers on which the individual chapters were based at the end still had my friend's name in it, so you could see it was a fake thesis). If a bona fide scientist makes a mistake about missing attribution, they would correct it as soon as possible. Many, however, would not correct such a re-discovery, because it's embarrassing. But the worst is when people don't even imagine anything like what they are working on could already exist, and they don't even bother finding and reading related work -- in other words, ignorance. Science deserves better, but there are more and more ignorant folks around that want to ignore all work before them.
Mond_ 1y ago
Oh boy, I sure can't wait to see the comments on this one! Schmidhuber sure seems to be a personality, and so far I've mostly heard negative things about his "I invented this" attitude to modern research.
- kleiba 1y ago
  A lot of this is because nobody likes braggers - however, in all fairness, his argument is that a lot of what is considered modern ML is based on many previous results, including but not limited to his own research.
goldemerald 1y ago
No discussion with Schmidhuber is complete without the infamous debate at NIPS 2016 https://youtu.be/HGYYEUSm-0Q?t=3780 https://youtu.be/HGYYEUSm-0Q?t=3780 . One of my goals as a ML researcher is to publish something and have Schmidhuber claim he's already done it. But more seriously, I'm not a fan of Schmidhuber because even if he truly did invent all this stuff early in the 90s, he's inability to see its application to modern compute held the field back by years. In principle, we could have had GANs and self-supervised models' years earlier if he had "revisited his early work". It's clear to me no one read his early paper's when developing GANs/self-supervision/transformers.
- deleted 1y ago
  [deleted]
- andy99 1y ago
  It's very common in science for people to have had results they didn't understand the significance of that later were popularized by someone else. There is the whole thing with Damadian claiming to have invented MRI (he didn't) when the Nobel prize went to Mansfield and Lauterbur (see the Nobel prize part of the article). https://en.m.wikipedia.org/wiki/Paul_Lauterbur https://en.m.wikipedia.org/wiki/Paul_Lauterbur And I've seen other less prominent examples. It's a lot like the difference between ideas and execution and people claiming someone "stole" their idea because they made a successful business from it.
- nextos 1y ago
  I think he did understand both the significance of his work and the importance of hardware. His group pioneered porting models to GPUs. But personal circumstances matter a lot. He was stuck at IDSIA in Lugano, i.e. relatively small and not-so-well funded academia. He could have done much better in industry, with access to lots of funding, a bigger headcount, and serious infrastructure. Ultimately, models matter much less than infrastructure. Transformers are not that important, other architectures such as deep SSMs or xLSTM are able to achieve comparable results.
- godelski 1y ago
  > if he had "revisited his early work". Given that you're a researcher yourself I'm surprised by this comment. Have you not yourself experienced the harsh rejection of "not novel"? That sounds like a great way to get stuck in review hell. (I know I've experienced this even when doing novel things just by too closely relating it to other methodologies when explaining "oh, it's just ____"). The other part seems weird too. Who isn't upset when their work doesn't get recognized and someone else gets credit. Are we not all human?
deleted 1y ago
[deleted]
deleted 1y ago
[deleted]
mindcrime 1y ago
I'll probably get flamed to death for saying this, but I like Jürgen. I mean, I don't know him in person (never met him) but I've seen a lot of his written work and interviews and what-not and he seems like an alright guy to me. Yes, I get it... there's that whole "ooooh, Jürgen is always trying to claim credit for everything" thing and all. But really, to me, it doesn't exactly come off that way. Note that he's often pointing out the lack of credit assigned even to people who lived and died centuries before him. His "shtick" to me isn't just about him saying "people didn't give me credit" but it seems more "AI people in general haven't credited the history of the field properly." And in many cases he seems to have a point.
- noosphr 1y ago
  It's a clash of cultures. He is an academic that cares for understanding where ideas came from. His detractors need to be the smartest people in the room to get paid millions and raise billions. It's not very sexy to say 'Oh yes, we are just using an old Soviet learning algorithm on better hardware. Turns out we would have lost the cold war if the USSR had access to a 5090.' , which won't get you the billions you need to build the supercomputers that push the state of the art today.
  - ks2048 1y ago
    It seems his "detractors" (or at least his foes) are also academics - i.e. the same culture - they just cite Hinton and LeCun instead of Schmidhuber.
    - noosphr 1y ago
      It helps your career to cite the head of AI in Facebook and the former head of Google. Not so much some academician who worked in the 1970s in the Soviet Socialist Republic of Kazakhstan.
      - ks2048 1y ago
        I believe the Schmidhuber-ignoring (according to him) began before those two were at Google/Meta. But, I suppose NYU/Bell Labs and U-Toronto will be more likely to be cited than somewhere in Munich or Switzerland...
FL33TW00D 1y ago
If you guys were the inventors of Facebook, you’d have invented Facebook
noosphr 1y ago
Before people say that he is claiming credit for things he didn't do, or that he invented everything, please read his own paper on the subject: https://people.idsia.ch/~juergen/deep-learning-history.html https://people.idsia.ch/~juergen/deep-learning-history.html The history section starts in 1676.
- swyx 1y ago
  > 1676: The Chain Rule For Backward Credit Assignment Schmidhuber is nothing but a stickler for backward credit assignment
ur-whale 1y ago
If there ever was an example of a terrible personality getting in the way of career, Schmidhuber is most definitely it. You may have had many brilliant ideas, but everyone makes an abrupt 180 when they see the tip your beard turn the corner at conferences, that can't be a good signal for getting awards.
- cycomanic 1y ago
  You do see how your statement is problematic when talking about scientific attribution and scientific prizes for achievements? I'd also point out that people that point out inconvenient truth are often not popular, that does not make them wrong though (in the general sense, I don't know enough about the foundations of ML to pass judgement here).
  - ur-whale 1y ago
    I most certainly hear you, but : you have a completely idealistic point of view. Which is a quasi-synonym for unrealistic. As much as we'd all love the real world to be simple, rational and just, it simply ain't, and on a very vast scale. As an instance of that general principle: people who do excellent work but lack basic social skills stay stuck 99% of the time. And history is littered with their anonymous corpses.
banq 1y ago
Why did Google give birth to the Transformer? Because Google created an ecosystem where everything could flourish, while the old man in Switzerland lacked such an environment—what could even the smartest and greatest individual do against that? As an organization, fostering an organically growing context is like governing a great nation with delicate care. A bottom-up (organic growth) environment is the core context for sustained innovation and development!
- ks2048 1y ago
  > Why did Google give birth to the Transformer? No, Schmidhuber gave birth to the transformer in 1991. https://x.com/SchmidhuberAI/status/1683870175299239937 https://x.com/SchmidhuberAI/status/1683870175299239937
esafak 1y ago
The fact that the field of machine learning keeps "discovering" things already established in other fields, and christening them with new names does lend some credence to Schmidhuber. The field is more industrial than academic, and cares about money more than credit, and industrial-scale data theft is all in a day's work. As another commenter said, his misfortune is being in a lab with no industrial affiliation.
voidhorse 1y ago
I haven't read the article or paper yet, but if the gist I'm getting from the comments is correct, Schmidhuber is generally correct about industry having horrible citation practices. I even see it at a small scale at work. People often fail or forget to mention the others that helped them generate their ideas. I would not be at all surprised if this behavior extended to research papers published by people in industry as opposed to academia. Good citation practice simply does not exist in industry. We're lucky in any of the thousand blog posts that reimplement some idea that was cranked out ages ago in academic circles are even aware of the original effort, let alone cite it. Citations are few and far between in industry literature generally. Obviously there are exceptions and this just my personal observation, I haven't done or found any kind of meta literary study illustrating such.
starchild3001 1y ago
He may be a Turing-award worthy researcher (hey he invented LSTM, and even I knew his name!) but modesty surely isn't his biggest strength :))
jandrewrogers 1y ago
As someone who was heavily involved in AI research in the 1990s, of a Schmidhuber flavored variety though I’ve never interacted with him, I do think there is some underlying truth to his general point. Most of the current theory isn’t actually new and at least some people should know better that are kind of pretending it was invented out of whole cloth recently. But at the time a lot of this was invented, the hardware simply wasn’t capable enough to reduce it to practice — it is why I got out — and a lot of people who are piling into the field are simultaneously disinterested in that history and claiming every idea that crosses their mind as a novel invention. This is not unique to AI. Many other subfields of computer science have a similar dynamic e.g. databases. I’ve seen people claiming novelty in ideas that were fully proven out in real systems in the 1970s (and abandoned, for good reason) but are oblivious to this fact because if it isn’t trivially discoverable on the Internet then it doesn’t exist. There is still a lot of interesting computer science research that only exists on physical paper, if you can find a copy. Maybe we should be better about digitizing the pre-Internet research but the value of that research isn’t always obvious and the terminology has changed. We don’t give enough credit to how clever some of those early researchers actually were, working from much more primitive foundations. Having gone spelunking a number of times into the old literature, it never ceases to amaze me the number of times I have found an insight that is neither known nor cited in modern literature. Literally lost knowledge. It is ironic that computer science, of all fields, should suffer from this.
- floren 1y ago
  You want to make your way in the Computer Science field? Simple. Calculate rough time of amnesia (hell, 10 years is plenty, probably 10 months is plenty), go to the dusty archives, dig out something fun, and go for it. It’s worked for many people, and it can work for you. -- ron minnich

115 points by kleiba 1y ago

I highly recommend going through his report; it is a solid basis for the curious to verify for themselves who invented what and when. Intellectual honesty is very important. That includes not selling something as new if there are already publications of the idea; worse than to hide one's knowledge of prior art is to not even look in the first place ("ostrich strategy").

careful_ai 1y ago

This post nails the common pain point: raw GenAI outputs feel detached from real codebases and impose an overhead that often negates any speed gains from AI. The idea that generative coding tools are "offloading the easy stuff while leaving the hard, domain-specific thinking to humans" mirrors my own experience. There’s a strange cognitive tax that comes with trying to validate AI-generated code—especially when it’s syntactically right but semantically wrong in subtle ways. What I’ve found most frustrating is that these tools often lack a sense of context. Not just in terms of project architecture, but also in terms of intent and long-term maintainability. I’ve ended up spending more time reviewing and rewriting than if I had just tackled it manually. Still, I’m hopeful. When paired with lightweight human-in-the-loop workflows—like using AI to explain legacy logic or spot inconsistencies—it can be a decent thought partner. But as a coder’s autopilot? Not there yet.

nharada 1y ago

Doesn’t he know the Turing Award is really just a generalization of the Fields Medal, an award that actually came years earlier?

logicchains 1y ago
I'm sure he wouldn't object to a Fields Medal either.
triceratops 1y ago
I chuckled but I also maybe didn't understand. Is the joke that computer science is a generalization of math? That can't be rigth.
- dgacmu 1y ago
  The joke is that schmidhuber is known for (rightly or wrongly) pointing to modern contributions in deep neural networks and saying they're just a trivial generalization/adaptation/etc. of work he did 30 years ago.

belval 1y ago

Every so often Schmidhuber is brought back to the front-page of HN, people will argue that he "invented it all" while others will say that he's a-posteriori claiming all the good ideas were his. Relativity Priority Dispute: https://en.wikipedia.org/wiki/Relativity_priority_dispute https://en.wikipedia.org/wiki/Relativity_priority_dispute We all stand on the shoulders of giants, things can be invented and reinvented and ideas can appear twice in a vacuum.

kleiba 1y ago
But as far as I understand, Schmidhuber's claim is more severe: namely that Bengio, Hinton and LeCun intentionally failed to cite prior work by others (including himself) but instead only cited each other in order to boost their respective scientific reputation. I personally think that he's not doing himself or his argument of favor by presenting it the way he does. While he basically argues that science should be totally objective and neutral, there's no denying that if you put yourself in a less likeable light, you're not going to make any friends. On the other hand, he's gone at length with compiling detailed references to support his points. I can appreciate that because it makes his argument a lot less hand-wavey: you can go to his blog and compare the cited references yourself. Except that I couldn't because I'm not an ML expert.
- jll29 1y ago
  I have seen many cases where people -- accidentally as well as intentionally -- copied or re-invented the work of others (a friend posted on LinkedIn that someone else plagiarized his whole Ph.D. thesis, including the title, so only the name was changed; only the references to seperately published papers on which the individual chapters were based at the end still had my friend's name in it, so you could see it was a fake thesis). If a bona fide scientist makes a mistake about missing attribution, they would correct it as soon as possible. Many, however, would not correct such a re-discovery, because it's embarrassing. But the worst is when people don't even imagine anything like what they are working on could already exist, and they don't even bother finding and reading related work -- in other words, ignorance. Science deserves better, but there are more and more ignorant folks around that want to ignore all work before them.

Mond_ 1y ago

Oh boy, I sure can't wait to see the comments on this one! Schmidhuber sure seems to be a personality, and so far I've mostly heard negative things about his "I invented this" attitude to modern research.

kleiba 1y ago
A lot of this is because nobody likes braggers - however, in all fairness, his argument is that a lot of what is considered modern ML is based on many previous results, including but not limited to his own research.

goldemerald 1y ago

No discussion with Schmidhuber is complete without the infamous debate at NIPS 2016 https://youtu.be/HGYYEUSm-0Q?t=3780 https://youtu.be/HGYYEUSm-0Q?t=3780 . One of my goals as a ML researcher is to publish something and have Schmidhuber claim he's already done it. But more seriously, I'm not a fan of Schmidhuber because even if he truly did invent all this stuff early in the 90s, he's inability to see its application to modern compute held the field back by years. In principle, we could have had GANs and self-supervised models' years earlier if he had "revisited his early work". It's clear to me no one read his early paper's when developing GANs/self-supervision/transformers.

deleted 1y ago
[deleted]
andy99 1y ago
It's very common in science for people to have had results they didn't understand the significance of that later were popularized by someone else. There is the whole thing with Damadian claiming to have invented MRI (he didn't) when the Nobel prize went to Mansfield and Lauterbur (see the Nobel prize part of the article). https://en.m.wikipedia.org/wiki/Paul_Lauterbur https://en.m.wikipedia.org/wiki/Paul_Lauterbur And I've seen other less prominent examples. It's a lot like the difference between ideas and execution and people claiming someone "stole" their idea because they made a successful business from it.
nextos 1y ago
I think he did understand both the significance of his work and the importance of hardware. His group pioneered porting models to GPUs. But personal circumstances matter a lot. He was stuck at IDSIA in Lugano, i.e. relatively small and not-so-well funded academia. He could have done much better in industry, with access to lots of funding, a bigger headcount, and serious infrastructure. Ultimately, models matter much less than infrastructure. Transformers are not that important, other architectures such as deep SSMs or xLSTM are able to achieve comparable results.
godelski 1y ago
> if he had "revisited his early work". Given that you're a researcher yourself I'm surprised by this comment. Have you not yourself experienced the harsh rejection of "not novel"? That sounds like a great way to get stuck in review hell. (I know I've experienced this even when doing novel things just by too closely relating it to other methodologies when explaining "oh, it's just ____"). The other part seems weird too. Who isn't upset when their work doesn't get recognized and someone else gets credit. Are we not all human?

deleted 1y ago

[deleted]

deleted 1y ago

[deleted]

mindcrime 1y ago

I'll probably get flamed to death for saying this, but I like Jürgen. I mean, I don't know him in person (never met him) but I've seen a lot of his written work and interviews and what-not and he seems like an alright guy to me. Yes, I get it... there's that whole "ooooh, Jürgen is always trying to claim credit for everything" thing and all. But really, to me, it doesn't exactly come off that way. Note that he's often pointing out the lack of credit assigned even to people who lived and died centuries before him. His "shtick" to me isn't just about him saying "people didn't give me credit" but it seems more "AI people in general haven't credited the history of the field properly." And in many cases he seems to have a point.

noosphr 1y ago
It's a clash of cultures. He is an academic that cares for understanding where ideas came from. His detractors need to be the smartest people in the room to get paid millions and raise billions. It's not very sexy to say 'Oh yes, we are just using an old Soviet learning algorithm on better hardware. Turns out we would have lost the cold war if the USSR had access to a 5090.' , which won't get you the billions you need to build the supercomputers that push the state of the art today.
- ks2048 1y ago
  It seems his "detractors" (or at least his foes) are also academics - i.e. the same culture - they just cite Hinton and LeCun instead of Schmidhuber.
  - noosphr 1y ago
    It helps your career to cite the head of AI in Facebook and the former head of Google. Not so much some academician who worked in the 1970s in the Soviet Socialist Republic of Kazakhstan.
    - ks2048 1y ago
      I believe the Schmidhuber-ignoring (according to him) began before those two were at Google/Meta. But, I suppose NYU/Bell Labs and U-Toronto will be more likely to be cited than somewhere in Munich or Switzerland...

FL33TW00D 1y ago

If you guys were the inventors of Facebook, you’d have invented Facebook

noosphr 1y ago

Before people say that he is claiming credit for things he didn't do, or that he invented everything, please read his own paper on the subject: https://people.idsia.ch/~juergen/deep-learning-history.html https://people.idsia.ch/~juergen/deep-learning-history.html The history section starts in 1676.

swyx 1y ago
> 1676: The Chain Rule For Backward Credit Assignment Schmidhuber is nothing but a stickler for backward credit assignment

ur-whale 1y ago

If there ever was an example of a terrible personality getting in the way of career, Schmidhuber is most definitely it. You may have had many brilliant ideas, but everyone makes an abrupt 180 when they see the tip your beard turn the corner at conferences, that can't be a good signal for getting awards.

cycomanic 1y ago
You do see how your statement is problematic when talking about scientific attribution and scientific prizes for achievements? I'd also point out that people that point out inconvenient truth are often not popular, that does not make them wrong though (in the general sense, I don't know enough about the foundations of ML to pass judgement here).
- ur-whale 1y ago
  I most certainly hear you, but : you have a completely idealistic point of view. Which is a quasi-synonym for unrealistic. As much as we'd all love the real world to be simple, rational and just, it simply ain't, and on a very vast scale. As an instance of that general principle: people who do excellent work but lack basic social skills stay stuck 99% of the time. And history is littered with their anonymous corpses.

banq 1y ago

Why did Google give birth to the Transformer? Because Google created an ecosystem where everything could flourish, while the old man in Switzerland lacked such an environment—what could even the smartest and greatest individual do against that? As an organization, fostering an organically growing context is like governing a great nation with delicate care. A bottom-up (organic growth) environment is the core context for sustained innovation and development!

ks2048 1y ago
> Why did Google give birth to the Transformer? No, Schmidhuber gave birth to the transformer in 1991. https://x.com/SchmidhuberAI/status/1683870175299239937 https://x.com/SchmidhuberAI/status/1683870175299239937

esafak 1y ago

The fact that the field of machine learning keeps "discovering" things already established in other fields, and christening them with new names does lend some credence to Schmidhuber. The field is more industrial than academic, and cares about money more than credit, and industrial-scale data theft is all in a day's work. As another commenter said, his misfortune is being in a lab with no industrial affiliation.

voidhorse 1y ago

I haven't read the article or paper yet, but if the gist I'm getting from the comments is correct, Schmidhuber is generally correct about industry having horrible citation practices. I even see it at a small scale at work. People often fail or forget to mention the others that helped them generate their ideas. I would not be at all surprised if this behavior extended to research papers published by people in industry as opposed to academia. Good citation practice simply does not exist in industry. We're lucky in any of the thousand blog posts that reimplement some idea that was cranked out ages ago in academic circles are even aware of the original effort, let alone cite it. Citations are few and far between in industry literature generally. Obviously there are exceptions and this just my personal observation, I haven't done or found any kind of meta literary study illustrating such.

starchild3001 1y ago

He may be a Turing-award worthy researcher (hey he invented LSTM, and even I knew his name!) but modesty surely isn't his biggest strength :))

jandrewrogers 1y ago

As someone who was heavily involved in AI research in the 1990s, of a Schmidhuber flavored variety though I’ve never interacted with him, I do think there is some underlying truth to his general point. Most of the current theory isn’t actually new and at least some people should know better that are kind of pretending it was invented out of whole cloth recently. But at the time a lot of this was invented, the hardware simply wasn’t capable enough to reduce it to practice — it is why I got out — and a lot of people who are piling into the field are simultaneously disinterested in that history and claiming every idea that crosses their mind as a novel invention. This is not unique to AI. Many other subfields of computer science have a similar dynamic e.g. databases. I’ve seen people claiming novelty in ideas that were fully proven out in real systems in the 1970s (and abandoned, for good reason) but are oblivious to this fact because if it isn’t trivially discoverable on the Internet then it doesn’t exist. There is still a lot of interesting computer science research that only exists on physical paper, if you can find a copy. Maybe we should be better about digitizing the pre-Internet research but the value of that research isn’t always obvious and the terminology has changed. We don’t give enough credit to how clever some of those early researchers actually were, working from much more primitive foundations. Having gone spelunking a number of times into the old literature, it never ceases to amaze me the number of times I have found an insight that is neither known nor cited in modern literature. Literally lost knowledge. It is ironic that computer science, of all fields, should suffer from this.

floren 1y ago
You want to make your way in the Computer Science field? Simple. Calculate rough time of amnesia (hell, 10 years is plenty, probably 10 months is plenty), go to the dusty archives, dig out something fun, and go for it. It’s worked for many people, and it can work for you. -- ron minnich