7 ms·

The AI dooming was never a thing for me. And I still don’t get it. I don’t see anything that would even point into that direction. Curious to understand where

by baxtr 1y ago

The AI dooming was never a thing for me. And I still don’t get it. I don’t see anything that would even point into that direction. Curious to understand where these thoughts are coming from

fastball 1y ago
https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a...
- habinero 1y ago
  That guy is so convinced he's a staggering genius and I have never understood why anyone else thinks it's true.
  - fastball 1y ago
    Possibly, but I do not think Yudkowsky's opinion of himself has any bearing on whether or not the above article is a good encapsulation of why some people are worried about AGI x-risk (and I think it is).
Sharlin 1y ago
Yes, fortunately these LLM things don't seem to be leading to anything that could be called an AGI. But that isn't saying that a real AGI capable of self-improvement couldn't be extremely dangerous.
GolDDranks 1y ago
> I don’t see anything that would even point into that direction. I find it a kind of baffling that people claim they can't see the problem. I'm not sure about the risk probabilities, but at least I can see that there clearly exists a potential problem. In a nutshell: Humans – the most intelligent species on the planet – have absolute power over any other species, specifically because of our intelligence and the accumulated technical prowess. Introducing another, equally or more intelligent thing into equation is going to risk that we end up with _not_ having the power over our existence.
- guiambros 1y ago
  Agree. I'm an AI optimist (mostly), but I find Richard Sutton's reasoning on this topic [1] very well argued. [1] https://youtu.be/FLOL2f4iHKA?si=Ot9EeiaF-68sSxkb https://youtu.be/FLOL2f4iHKA?si=Ot9EeiaF-68sSxkb
- NL807 1y ago
  Lot of doomers gloss over the fact that AI is bounded by the laws of physics, raw resources, energy and the monumental cost of reproducing them. Humans can reproduce by simply having sex, eating food and drinking water. AI can reproduce by first mining resources, refining said resources, building another Shenzhen, then rolling out another fab at the same scale of TSMC. That is assuming the AI wants control over the entire process. This kind of logistics requires cooperation of an entire civilisation. Any attempt by an AI could be trivially stopped because of the large scope of the infrastructure required.
  - olddustytrail 1y ago
    Sure, trivially. Let's see you do it then. There are new data centres being built and that's just for LLMs. So stop them. Are you starting to see the problem? You might want to stop a rogue AI but you can bet there will be someone else who thinks it will make them rich, or powerful, or they just want to see the world burn.
    - NL807 1y ago
      >You might want to stop a rogue AI but you can bet there will be someone else who thinks it will make them rich, or powerful, or they just want to see the world burn. What makes you think they will not be stopped? This one guy needs a dedicated power plant, an entire data centre, and need to source all the components and materials to build it. Again. Heavy reliance on logistics and supply chain. He can't possibly control all of those, and disrupting just a few (which would be easy) will inevitably prevent him and his AI progressing any further. At best, he'd be a mad king and his machine pet trapped in a castle, surrounded by a world that is turned against him. His days would be almost certainly numbered.
parineum 1y ago
> Curious to understand where these thoughts are coming from It's a cynical take but all this AGI talk seems to be driven by either CEOs of companies with a financial interest in the hype or prominent intellectuals with a financial interest in the doom and gloom. Sam Altman and Sam Harris can pit themselves against each other and, as long as everyone is watching the ping pong ball back and forth, they both win.
Sankozi 1y ago
More intelligent specie (AI) designed by specie (humans) that has history of eradicating less intelligent species (neanderthals). I don't see how anyone can't see the problem.

by baxtr 1y ago

The AI dooming was never a thing for me. And I still don’t get it. I don’t see anything that would even point into that direction. Curious to understand where these thoughts are coming from

fastball 1y ago

https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a...

habinero 1y ago
That guy is so convinced he's a staggering genius and I have never understood why anyone else thinks it's true.
- fastball 1y ago
  Possibly, but I do not think Yudkowsky's opinion of himself has any bearing on whether or not the above article is a good encapsulation of why some people are worried about AGI x-risk (and I think it is).

Sharlin 1y ago

Yes, fortunately these LLM things don't seem to be leading to anything that could be called an AGI. But that isn't saying that a real AGI capable of self-improvement couldn't be extremely dangerous.

GolDDranks 1y ago

> I don’t see anything that would even point into that direction. I find it a kind of baffling that people claim they can't see the problem. I'm not sure about the risk probabilities, but at least I can see that there clearly exists a potential problem. In a nutshell: Humans – the most intelligent species on the planet – have absolute power over any other species, specifically because of our intelligence and the accumulated technical prowess. Introducing another, equally or more intelligent thing into equation is going to risk that we end up with _not_ having the power over our existence.

guiambros 1y ago
Agree. I'm an AI optimist (mostly), but I find Richard Sutton's reasoning on this topic [1] very well argued. [1] https://youtu.be/FLOL2f4iHKA?si=Ot9EeiaF-68sSxkb https://youtu.be/FLOL2f4iHKA?si=Ot9EeiaF-68sSxkb
NL807 1y ago
Lot of doomers gloss over the fact that AI is bounded by the laws of physics, raw resources, energy and the monumental cost of reproducing them. Humans can reproduce by simply having sex, eating food and drinking water. AI can reproduce by first mining resources, refining said resources, building another Shenzhen, then rolling out another fab at the same scale of TSMC. That is assuming the AI wants control over the entire process. This kind of logistics requires cooperation of an entire civilisation. Any attempt by an AI could be trivially stopped because of the large scope of the infrastructure required.
- olddustytrail 1y ago
  Sure, trivially. Let's see you do it then. There are new data centres being built and that's just for LLMs. So stop them. Are you starting to see the problem? You might want to stop a rogue AI but you can bet there will be someone else who thinks it will make them rich, or powerful, or they just want to see the world burn.
  - NL807 1y ago
    >You might want to stop a rogue AI but you can bet there will be someone else who thinks it will make them rich, or powerful, or they just want to see the world burn. What makes you think they will not be stopped? This one guy needs a dedicated power plant, an entire data centre, and need to source all the components and materials to build it. Again. Heavy reliance on logistics and supply chain. He can't possibly control all of those, and disrupting just a few (which would be easy) will inevitably prevent him and his AI progressing any further. At best, he'd be a mad king and his machine pet trapped in a castle, surrounded by a world that is turned against him. His days would be almost certainly numbered.

parineum 1y ago

> Curious to understand where these thoughts are coming from It's a cynical take but all this AGI talk seems to be driven by either CEOs of companies with a financial interest in the hype or prominent intellectuals with a financial interest in the doom and gloom. Sam Altman and Sam Harris can pit themselves against each other and, as long as everyone is watching the ping pong ball back and forth, they both win.

Sankozi 1y ago

More intelligent specie (AI) designed by specie (humans) that has history of eradicating less intelligent species (neanderthals). I don't see how anyone can't see the problem.