5 ms·

What made Hinton into an AI doomer

25 points by elsewhen 3y ago

villgax 3y ago
Man went off the deep end
lisasays 3y ago
https://archive.ph/Ji7IR https://archive.ph/Ji7IR Recent discussion of Google's PaLM: https://news.ycombinator.com/item?id=30908941 https://news.ycombinator.com/item?id=30908941
- revelio 3y ago
  A year ago, not so recent in AI time. The top comment there is quite amusing given how things have played out.
  - lisasays 3y ago
    Yeah it was late, sorry. If you don't mind: which comment are you referring to? (Seems we have a different top by now).
    - revelio 3y ago
      This one: "This is why Google built TPUs. This alone justifies the whole program by itself. This level of natural language understanding, once it is harnessed for applications and made efficient enough for wide use, is going to revolutionize literally everything Google does. Owning the chips that can do this is incredibly valuable and companies that are stuck purchasing or renting whatever Nvidia makes are going to be at a disadvantage." The last thing I read about that was that nVidia hardware is better than TPUs now, those chips don't seem to be a particular competitive advantage (any more?) and all the AI startups seem to be using Azure anyway. Things change fast.
ftxbro 3y ago
I can't access the wired article but in one interview he had an angle that I thought was interesting. It was new to me, which was weird because I saw so many doomer angles. His bold angle was that the automatic differentiability makes artificial optimization (training) so much more effective than any algorithm that biological neural systems might be using. According to wikipedia neural systems can do some backpropagation (https://en.wikipedia.org/wiki/Neural_backpropagation https://en.wikipedia.org/wiki/Neural_backpropagation) but Hinton was suggesting that the artificial optimization methods can be so much more powerful. This was funny to me because usually people allow biology the benefit of the doubt for the sake of argument, meekly begging to assume that artificial training might approach the biological abilities to within some order of magnitude, and submissively suggesting that one artificial neural parameter might be worth like 1/100 of a synapse or so if it would please the biologists. Then the AI researchers would say something like even if Moore's law diminishes we will still reach human power by 2050 or whatever. But not Hinton. He's like yeah the artificial training uses math power that goes beyond what our squishy neurotransmitters can muster, and if I were allowed to say AGI already exists then I'd say it, but I'm not allowed so instead I'll say AGI within five years.
- TechnicolorByte 3y ago
  Having a tenuous grasp of how modern AI/DL networks work and an even more tenuous grasp of neuroscience, can I ask for a ELI5 of the above and a source? Are you saying that Hinton is not a doomer and believes that AGI is inevitable very soon?
  - ALittleLight 3y ago
    How does the parent comment imply Hinton is not a doomer? AI Doom is the idea that AI will be very bad for humanity (e.g. extinction) and the idea that AGI is near is compatible with that. To my knowledge Hinton thinks Doom is a significant possibility and that AGI is plausibly near.
    - svnt 3y ago
      Using the term doomer generally implies the doom is unwarranted/an overreaction. Parent is asking whether GP thinks Hinton is justified in predicting actual doom.
haha69 3y ago
This was the title of the article: "What Really Made Geoffrey Hinton Into an AI Doomer" He is not an AI doomer? Talking about the dangers of AI is not the same as being a doomer. He also has positive things to say about it. A few tweets to qualify my claims 1. https://twitter.com/geoffreyhinton/status/1635739459764322330 https://twitter.com/geoffreyhinton/status/163573945976432233... 2. https://twitter.com/geoffreyhinton/status/1654572133744345088 https://twitter.com/geoffreyhinton/status/165457213374434508... 3. https://twitter.com/geoffreyhinton/status/1652993570721210372 https://twitter.com/geoffreyhinton/status/165299357072121037... And I see that even the article quotes anecdotes of him not being a "doomer". So I guess the headline is intentional and clickbait
- ftxbro 3y ago
  I guess it depends what you mean by a "doomer." If, to you, "doomer" means you have to act like Yudkowsky and go around saying it's urgent to unilaterally destroy all GPU data centers with military force then no he's not a doomer. But if "doomer" is allowed to include a Cambridge educated professor and Fellow of the Royal Society with a 'stiff upper lip' who finds himself surprised to realize that the caterpillar of humanity is creating the butterfly of AI and that "the caterpillar gets converted into a soup out of which the butterfly is created" as quoted in the first tweet thread that you listed, then maybe he could deserve the doomer label. In another interview, in his response to 'why we should be scared' asked by a reporter he said that "as a friend of mine said, it’s as if some genetic engineers said, we’re going to improve grizzly bears; we’ve already improved them with an IQ of 65, and they can talk English now, and they’re very useful for all sorts of things, but we think we can improve the IQ to 210." Presumably this guy is understated enough to trust the listener to understand that he thinks that might reasonably scare someone.
- revelio 3y ago
  Of course he's a doomer. He's compared himself to Robert Oppenheimer and by his own admission the only reason he isn't calling for a complete halt to all AI research is because he's too pessimistic it'd be possible. He describes a man who wants to literally bomb datacenters as "not crazy", just "not helpful" i.e. he'd be totally on board with such tactics if he thought they were more plausible. > Talking about the dangers of AI is not the same as being a doomer It is in fact the same thing as being a doomer if you're claiming that something unstoppable will lead to doom, in the same way that pointing out government coverups of lab leaks technically makes you a conspiracy theorist.
tonjin12 3y ago
[dead]
revelio 3y ago
Here's a rewritten headline, definitely not ChatGPT approved: "Salty ex-Googler tries to slow down competition by mounting misinformation campaign in press". What a ridiculous charade. For years Google reacted to every OpenAI announcement or API launch by doing a blog post stating that they had already done it, their models were much better and they weren't releasing any evidence because their tech wasn't "safe" enough to make available to anyone, except, um, the benignly enlightened employees of Google themselves. And for years, like a fool, I believed them when they said their tech was better. ChatGPT ripped the mask off, letting everyone see this supposed danger they had to be protected from for themselves. And it's laughable. Then Bard launched and we learned that Team Hinton's models weren't really better at all, that whole message was corporate misinformation or maybe self delusion or maybe both. Then it became "PaLM is definitely GPT-4 quality, we just aren't making it available yet". Uh huh. Now we have this even more pathetic sight. Hinton, trying to set himself up as a classic public intellectual by extrapolating a trend into infinity until it reaches an end of the world that might happen "5-20" years from now, citing as evidence Proof By Snoop Dogg. He can go over there in the corner along with the "world hunger through overpopulation" and "New York under water by 2010" people. Having spent a lifetime chasing a goal he clearly never expected nor even wanted to achieve, he's now been lapped by his own former student who was busy actually making the AI revolution happen whilst Googlers were telling each other how virtuous they were. And oh yes, these supposed great dangers. What are they, according to Hinton? Ourselves. Like all left wing purity spirallers, his biggest fear is that people like him lose their power over politics and information. We might believe things written by an AI instead of journalists. These people claim to live in fear of misinformation, but it's not credible. Where were they when US media was waging an industrial-scale misinformation campaign about non-existent Russian AI bots on Twitter? The ones who were supposedly secretly fielding GPT-4 quality bots to make Americans vote for Trump? So many claims, all fallen, and yet these brave anti-disinfo warriors so concerned about elections were nowhere to be found at the time. Every time another round in this anti-progress, "let us smart people control things" salvo from Hinton appears, I breathe a deep sigh of relief that Team Sutskever exists and is doing so well. Left to their own devices Googlers would have happily sat around writing blog posts about how dangerous we all are forever. And I say that, sadly, as an ex Googler.
- 3y ago
xg15 3y ago
> “And I’m like, ‘Is we in a fucking movie right now or what?’” I mean, you could probably ask that question since at least 2016.
ilaksh 3y ago
GPT-4 is not alive or anything like that but it is intelligent. And it's approximately human level now. Computer hardware historically improves exponentially. We should anticipate a system that thinks 100 times faster than a human in a few years or less. This is barely even speculative. Two orders of magnitude performance improvement has been achieved many, many times over in the history of computing. This is a very specific application and there is still probably relatively low hanging fruit for optimizing it with hardware and software customization. New compute-in-memory paradigms are coming soon. The problem with hyperspeed GPT is that in an environment where companies and countries are competing, there is a strong incentive to give the AI very broad goals. Because having it wait a day for the humans to make decisions would mean 100 days of progress made by competitors (if it is at that speed). The human operators are effectively frozen relative to the activity of this speed of agent. They become speculators on the sidelines for the most part. Also, you can't prevent people from instructing it to pursue its own selfish goals or having self-preservation or reproductive (copying code/model weights) goals. These will be open source models. I think the only answer is that you ban the manufacture of AI hardware that exceeds a certain level of performance relative to humans. Also immediately start strongly discouraging people from simulating lifelike qualities in intelligent digital systems.

25 points by elsewhen 3y ago

Man went off the deep end

https://archive.ph/Ji7IR https://archive.ph/Ji7IR Recent discussion of Google's PaLM: https://news.ycombinator.com/item?id=30908941 https://news.ycombinator.com/item?id=30908941

revelio 3y ago
A year ago, not so recent in AI time. The top comment there is quite amusing given how things have played out.
- lisasays 3y ago
  Yeah it was late, sorry. If you don't mind: which comment are you referring to? (Seems we have a different top by now).
  - revelio 3y ago
    This one: "This is why Google built TPUs. This alone justifies the whole program by itself. This level of natural language understanding, once it is harnessed for applications and made efficient enough for wide use, is going to revolutionize literally everything Google does. Owning the chips that can do this is incredibly valuable and companies that are stuck purchasing or renting whatever Nvidia makes are going to be at a disadvantage." The last thing I read about that was that nVidia hardware is better than TPUs now, those chips don't seem to be a particular competitive advantage (any more?) and all the AI startups seem to be using Azure anyway. Things change fast.

ftxbro 3y ago

I can't access the wired article but in one interview he had an angle that I thought was interesting. It was new to me, which was weird because I saw so many doomer angles. His bold angle was that the automatic differentiability makes artificial optimization (training) so much more effective than any algorithm that biological neural systems might be using. According to wikipedia neural systems can do some backpropagation (https://en.wikipedia.org/wiki/Neural_backpropagation https://en.wikipedia.org/wiki/Neural_backpropagation) but Hinton was suggesting that the artificial optimization methods can be so much more powerful. This was funny to me because usually people allow biology the benefit of the doubt for the sake of argument, meekly begging to assume that artificial training might approach the biological abilities to within some order of magnitude, and submissively suggesting that one artificial neural parameter might be worth like 1/100 of a synapse or so if it would please the biologists. Then the AI researchers would say something like even if Moore's law diminishes we will still reach human power by 2050 or whatever. But not Hinton. He's like yeah the artificial training uses math power that goes beyond what our squishy neurotransmitters can muster, and if I were allowed to say AGI already exists then I'd say it, but I'm not allowed so instead I'll say AGI within five years.

TechnicolorByte 3y ago
Having a tenuous grasp of how modern AI/DL networks work and an even more tenuous grasp of neuroscience, can I ask for a ELI5 of the above and a source? Are you saying that Hinton is not a doomer and believes that AGI is inevitable very soon?
- ALittleLight 3y ago
  How does the parent comment imply Hinton is not a doomer? AI Doom is the idea that AI will be very bad for humanity (e.g. extinction) and the idea that AGI is near is compatible with that. To my knowledge Hinton thinks Doom is a significant possibility and that AGI is plausibly near.
  - svnt 3y ago
    Using the term doomer generally implies the doom is unwarranted/an overreaction. Parent is asking whether GP thinks Hinton is justified in predicting actual doom.

haha69 3y ago

This was the title of the article: "What Really Made Geoffrey Hinton Into an AI Doomer" He is not an AI doomer? Talking about the dangers of AI is not the same as being a doomer. He also has positive things to say about it. A few tweets to qualify my claims 1. https://twitter.com/geoffreyhinton/status/1635739459764322330 https://twitter.com/geoffreyhinton/status/163573945976432233... 2. https://twitter.com/geoffreyhinton/status/1654572133744345088 https://twitter.com/geoffreyhinton/status/165457213374434508... 3. https://twitter.com/geoffreyhinton/status/1652993570721210372 https://twitter.com/geoffreyhinton/status/165299357072121037... And I see that even the article quotes anecdotes of him not being a "doomer". So I guess the headline is intentional and clickbait

ftxbro 3y ago
I guess it depends what you mean by a "doomer." If, to you, "doomer" means you have to act like Yudkowsky and go around saying it's urgent to unilaterally destroy all GPU data centers with military force then no he's not a doomer. But if "doomer" is allowed to include a Cambridge educated professor and Fellow of the Royal Society with a 'stiff upper lip' who finds himself surprised to realize that the caterpillar of humanity is creating the butterfly of AI and that "the caterpillar gets converted into a soup out of which the butterfly is created" as quoted in the first tweet thread that you listed, then maybe he could deserve the doomer label. In another interview, in his response to 'why we should be scared' asked by a reporter he said that "as a friend of mine said, it’s as if some genetic engineers said, we’re going to improve grizzly bears; we’ve already improved them with an IQ of 65, and they can talk English now, and they’re very useful for all sorts of things, but we think we can improve the IQ to 210." Presumably this guy is understated enough to trust the listener to understand that he thinks that might reasonably scare someone.
revelio 3y ago
Of course he's a doomer. He's compared himself to Robert Oppenheimer and by his own admission the only reason he isn't calling for a complete halt to all AI research is because he's too pessimistic it'd be possible. He describes a man who wants to literally bomb datacenters as "not crazy", just "not helpful" i.e. he'd be totally on board with such tactics if he thought they were more plausible. > Talking about the dangers of AI is not the same as being a doomer It is in fact the same thing as being a doomer if you're claiming that something unstoppable will lead to doom, in the same way that pointing out government coverups of lab leaks technically makes you a conspiracy theorist.

tonjin12 3y ago

[dead]

revelio 3y ago

Here's a rewritten headline, definitely not ChatGPT approved: "Salty ex-Googler tries to slow down competition by mounting misinformation campaign in press". What a ridiculous charade. For years Google reacted to every OpenAI announcement or API launch by doing a blog post stating that they had already done it, their models were much better and they weren't releasing any evidence because their tech wasn't "safe" enough to make available to anyone, except, um, the benignly enlightened employees of Google themselves. And for years, like a fool, I believed them when they said their tech was better. ChatGPT ripped the mask off, letting everyone see this supposed danger they had to be protected from for themselves. And it's laughable. Then Bard launched and we learned that Team Hinton's models weren't really better at all, that whole message was corporate misinformation or maybe self delusion or maybe both. Then it became "PaLM is definitely GPT-4 quality, we just aren't making it available yet". Uh huh. Now we have this even more pathetic sight. Hinton, trying to set himself up as a classic public intellectual by extrapolating a trend into infinity until it reaches an end of the world that might happen "5-20" years from now, citing as evidence Proof By Snoop Dogg. He can go over there in the corner along with the "world hunger through overpopulation" and "New York under water by 2010" people. Having spent a lifetime chasing a goal he clearly never expected nor even wanted to achieve, he's now been lapped by his own former student who was busy actually making the AI revolution happen whilst Googlers were telling each other how virtuous they were. And oh yes, these supposed great dangers. What are they, according to Hinton? Ourselves. Like all left wing purity spirallers, his biggest fear is that people like him lose their power over politics and information. We might believe things written by an AI instead of journalists. These people claim to live in fear of misinformation, but it's not credible. Where were they when US media was waging an industrial-scale misinformation campaign about non-existent Russian AI bots on Twitter? The ones who were supposedly secretly fielding GPT-4 quality bots to make Americans vote for Trump? So many claims, all fallen, and yet these brave anti-disinfo warriors so concerned about elections were nowhere to be found at the time. Every time another round in this anti-progress, "let us smart people control things" salvo from Hinton appears, I breathe a deep sigh of relief that Team Sutskever exists and is doing so well. Left to their own devices Googlers would have happily sat around writing blog posts about how dangerous we all are forever. And I say that, sadly, as an ex Googler.

3y ago

xg15 3y ago

> “And I’m like, ‘Is we in a fucking movie right now or what?’” I mean, you could probably ask that question since at least 2016.

ilaksh 3y ago

GPT-4 is not alive or anything like that but it is intelligent. And it's approximately human level now. Computer hardware historically improves exponentially. We should anticipate a system that thinks 100 times faster than a human in a few years or less. This is barely even speculative. Two orders of magnitude performance improvement has been achieved many, many times over in the history of computing. This is a very specific application and there is still probably relatively low hanging fruit for optimizing it with hardware and software customization. New compute-in-memory paradigms are coming soon. The problem with hyperspeed GPT is that in an environment where companies and countries are competing, there is a strong incentive to give the AI very broad goals. Because having it wait a day for the humans to make decisions would mean 100 days of progress made by competitors (if it is at that speed). The human operators are effectively frozen relative to the activity of this speed of agent. They become speculators on the sidelines for the most part. Also, you can't prevent people from instructing it to pursue its own selfish goals or having self-preservation or reproductive (copying code/model weights) goals. These will be open source models. I think the only answer is that you ban the manufacture of AI hardware that exceeds a certain level of performance relative to humans. Also immediately start strongly discouraging people from simulating lifelike qualities in intelligent digital systems.