6 ms·

Will people finally wake up that the AGI X-Risk people have been right and we’re rapidly approaching a really fucking big deal? This forum has been so behind f

Will people finally wake up that the AGI X-Risk people have been right and we’re rapidly approaching a really fucking big deal? This forum has been so behind for too long. Sama has been saying this a decade now: “Development of Superhuman machine intelligence is probably the greatest threat to the continued existence of humanity” 2015 https://blog.samaltman.com/machine-intelligence-part-1 https://blog.samaltman.com/machine-intelligence-part-1 Hinton, Ilya, Dario Amodei, RLHF inventor, Deepmind founders. They all get it, which is why they’re the smart cookies in those positions. First stage is denial, I get it, not easy to swallow the gravity of what’s coming.

ffsm8 2y ago
People have been predicting the singularity to occur sometimes around 2030 and 2045 waaaay further back then 2015. And not just by enthusiasts, I dimly remember an interview with Richard Darkins from back in the day... Though that doesn't mean that the current version of language models will ever achieve AGI, and I sincerely doubt they will. They'll likely be a component in the AI, but likely not the thing that "drives"
- neural_thing 2y ago
  Vernor Vinge as much as anyone can be credited with the concept of the singularity. In his 1993 essay on it, he said he'd be surprised if it happened before 2005 or after 2030 https://edoras.sdsu.edu/~vinge/misc/singularity.html https://edoras.sdsu.edu/~vinge/misc/singularity.html
  - ffsm8 2y ago
    Fwiw, that prediction was during Moore's law though. If that held until now, CPUs would run laps around what our current gpus do for LLMs.
archagon 2y ago
And why are Altman's words worth anything? Is he some sort of great thinker? Or a leading AI researcher, perhaps? No. Altman is in his current position because he's highly effective at consolidating power and has friends in high places. That's it. Everything he says can be seen as marketing for the next power grab.
- skeeter2020 2y ago
  well, he did also have a an early (failed) YC startup - does that add cred?
- tim333 2y ago
  Altman did play some part in bringing ChatGPT about. I think the point is the people making AI or running companies making current AI are saying be wary. In general it's worth weighting the opinions of people who are leaders in a field, about that field, over people who know little about it.
hn_throwaway_99 2y ago
> Will people finally wake up that the AGI X-Risk people have been right and we’re rapidly approaching a really fucking big deal? OK, say I totally believe this. What, pray tell, are we supposed to do about it? Don't you at least see the irony of quoting Sama's dire warnings about the development of AI, without at least mentioning that he is at the absolute forefront of the push to build this technology that can destroy all of humanity. It's like he's saying "This potion can destroy all of humanity if we make it" as he works faster and faster to figure out how to make it. I mean, I get it, "if we don't build it, someone else will", but all of the discussion around "alignment" seems just blatantly laughable to me. If on one hand your goal is to build "super intelligence", i.e. way smarter than any human or group of humans, how do you expect to control that super intelligence when you're just acting at the middling level of human intelligence? While I'm skeptical on the timeline, if we do ever end up building super intelligence, the idea that we can control it is a pipe dream. We may not be toast (I mean, we're smarter than dogs, and we keep them around), but we won't be in control. So if you truly believe super intelligent AI is coming, you may as well enjoy the view now, because there ain't nothing you or anyone else will be able to do to "save humanity" if or when it arrives.
- achierius 2y ago
  Political organization to force a stop to ongoing research? Protest outside OAI HQ? There are lots of thing we could, and many of us would, do if more people were actually convinced their life were in danger.
  - hn_throwaway_99 2y ago
    > Political organization to force a stop to ongoing research? Protest outside OAI HQ? Come on, be real. Do you honestly think that would make a lick of difference? Maybe, at best, delay things by a couple months. But this is a worldwide phenomenon, and humans have shown time and time again that they are not able to self organize globally. How successful do you think that political organization is going to be in slowing China's progress?
    - 2y ago
pixl97 2y ago
>This forum has been so behind for too long. There is a strong financial incentive for a lot of people on this site to deny they are at risk from it, or to deny what they are building has risk and they should have culpability from that.
samr71 2y ago
It's not something you need to worry about. If we get the Singularity, it's overwhelmingly likely Jesus will return concurrently.
- tim333 2y ago
  Though possibly only in AI form.
goatlover 2y ago
> "Development of Superhuman machine intelligence is probably the greatest threat to the continued existence of humanity” If that's really true, why is there such a big push to rapidly improve AI? I'm guessing OpenAI, Google, Anthropic, Apple, Meta, Boston Dynamics don't really believe this. They believe AI will make them billions. What is OpenAI's definition of AGI? A model that makes $100 billion?
- AgentME 2y ago
  Because they also believe the development of superhuman machine intelligence will probably be the greatest invention for humanity. The possible upsides and downsides are both staggeringly huge and uncertain.
- medvezhenok 2y ago
  You can also have prisoner’s dilemma where no single actor is capable of stopping AI’s advance

ffsm8 2y ago

People have been predicting the singularity to occur sometimes around 2030 and 2045 waaaay further back then 2015. And not just by enthusiasts, I dimly remember an interview with Richard Darkins from back in the day... Though that doesn't mean that the current version of language models will ever achieve AGI, and I sincerely doubt they will. They'll likely be a component in the AI, but likely not the thing that "drives"

neural_thing 2y ago
Vernor Vinge as much as anyone can be credited with the concept of the singularity. In his 1993 essay on it, he said he'd be surprised if it happened before 2005 or after 2030 https://edoras.sdsu.edu/~vinge/misc/singularity.html https://edoras.sdsu.edu/~vinge/misc/singularity.html
- ffsm8 2y ago
  Fwiw, that prediction was during Moore's law though. If that held until now, CPUs would run laps around what our current gpus do for LLMs.

archagon 2y ago

And why are Altman's words worth anything? Is he some sort of great thinker? Or a leading AI researcher, perhaps? No. Altman is in his current position because he's highly effective at consolidating power and has friends in high places. That's it. Everything he says can be seen as marketing for the next power grab.

skeeter2020 2y ago
well, he did also have a an early (failed) YC startup - does that add cred?
tim333 2y ago
Altman did play some part in bringing ChatGPT about. I think the point is the people making AI or running companies making current AI are saying be wary. In general it's worth weighting the opinions of people who are leaders in a field, about that field, over people who know little about it.

hn_throwaway_99 2y ago

> Will people finally wake up that the AGI X-Risk people have been right and we’re rapidly approaching a really fucking big deal? OK, say I totally believe this. What, pray tell, are we supposed to do about it? Don't you at least see the irony of quoting Sama's dire warnings about the development of AI, without at least mentioning that he is at the absolute forefront of the push to build this technology that can destroy all of humanity. It's like he's saying "This potion can destroy all of humanity if we make it" as he works faster and faster to figure out how to make it. I mean, I get it, "if we don't build it, someone else will", but all of the discussion around "alignment" seems just blatantly laughable to me. If on one hand your goal is to build "super intelligence", i.e. way smarter than any human or group of humans, how do you expect to control that super intelligence when you're just acting at the middling level of human intelligence? While I'm skeptical on the timeline, if we do ever end up building super intelligence, the idea that we can control it is a pipe dream. We may not be toast (I mean, we're smarter than dogs, and we keep them around), but we won't be in control. So if you truly believe super intelligent AI is coming, you may as well enjoy the view now, because there ain't nothing you or anyone else will be able to do to "save humanity" if or when it arrives.

achierius 2y ago
Political organization to force a stop to ongoing research? Protest outside OAI HQ? There are lots of thing we could, and many of us would, do if more people were actually convinced their life were in danger.
- hn_throwaway_99 2y ago
  > Political organization to force a stop to ongoing research? Protest outside OAI HQ? Come on, be real. Do you honestly think that would make a lick of difference? Maybe, at best, delay things by a couple months. But this is a worldwide phenomenon, and humans have shown time and time again that they are not able to self organize globally. How successful do you think that political organization is going to be in slowing China's progress?
  - 2y ago

pixl97 2y ago

>This forum has been so behind for too long. There is a strong financial incentive for a lot of people on this site to deny they are at risk from it, or to deny what they are building has risk and they should have culpability from that.

samr71 2y ago

It's not something you need to worry about. If we get the Singularity, it's overwhelmingly likely Jesus will return concurrently.

tim333 2y ago
Though possibly only in AI form.

goatlover 2y ago

> "Development of Superhuman machine intelligence is probably the greatest threat to the continued existence of humanity” If that's really true, why is there such a big push to rapidly improve AI? I'm guessing OpenAI, Google, Anthropic, Apple, Meta, Boston Dynamics don't really believe this. They believe AI will make them billions. What is OpenAI's definition of AGI? A model that makes $100 billion?

AgentME 2y ago
Because they also believe the development of superhuman machine intelligence will probably be the greatest invention for humanity. The possible upsides and downsides are both staggeringly huge and uncertain.
medvezhenok 2y ago
You can also have prisoner’s dilemma where no single actor is capable of stopping AI’s advance