3 ms·

Why? There’s nothing in the current process of developing AI that would lead to a AI that would act against humanity of its own choosing. The development proces

by audunw 1y ago

Why? There’s nothing in the current process of developing AI that would lead to a AI that would act against humanity of its own choosing. The development process is hyper-optimised to make AIs that do exactly what humans tell it to do. Sure, an LLM AI can role-play as an evil super AI out to kill humans. But it can just as easily role-play as one that defends humanity. So that tells us nothing about what will happen. We could just as well think that exploding the first nuclear bomb would ignite the atmosphere and kill all of humanity. There was nothing from physics that indicated it was possible but some still thought about it. IMO that kind of thinking is pointless. Same with thinking LHC would create a black hole. As far as I can tell the fear of super intelligent AI will kill humans all boils down to something utterly magical happens and then somehow super intelligent evil AI appears.

myrmidon 1y ago
> Why? There’s nothing in the current process of developing AI that would lead to a AI that would act against humanity of its own choosing. If we had certainty that our designs were categorically incapable of acting in their own interest then I would agree with you, but we absolutely don't, and I'd argue that we don't even have that certainty for current-generation LLMs. Long term, we're fundamentally competing with AI for ressources. > We could just as well think that exploding the first nuclear bomb would ignite the atmosphere and kill all of humanity. There was nothing from physics that indicated it was possible but some still thought about it. This is incorrect. Atmospheric ignition was a concern by nuclear physicists based on physics, but dismissed as unlikely after doing the math (see https://en.wikipedia.org/wiki/Effects_of_nuclear_explosions#Atmospheric_ignition https://en.wikipedia.org/wiki/Effects_of_nuclear_explosions#...). > As far as I can tell the fear of super intelligent AI will kill humans all boils down to something utterly magical happens and then somehow super intelligent evil AI appears. Not necessary at all. AI acting in its own interests and competing "honestly" with humans is already enough. This is exactly how we outcompeted every other animal on the planet after all.

by audunw 1y ago

myrmidon 1y ago

> Why? There’s nothing in the current process of developing AI that would lead to a AI that would act against humanity of its own choosing. If we had certainty that our designs were categorically incapable of acting in their own interest then I would agree with you, but we absolutely don't, and I'd argue that we don't even have that certainty for current-generation LLMs. Long term, we're fundamentally competing with AI for ressources. > We could just as well think that exploding the first nuclear bomb would ignite the atmosphere and kill all of humanity. There was nothing from physics that indicated it was possible but some still thought about it. This is incorrect. Atmospheric ignition was a concern by nuclear physicists based on physics, but dismissed as unlikely after doing the math (see https://en.wikipedia.org/wiki/Effects_of_nuclear_explosions#Atmospheric_ignition https://en.wikipedia.org/wiki/Effects_of_nuclear_explosions#...). > As far as I can tell the fear of super intelligent AI will kill humans all boils down to something utterly magical happens and then somehow super intelligent evil AI appears. Not necessary at all. AI acting in its own interests and competing "honestly" with humans is already enough. This is exactly how we outcompeted every other animal on the planet after all.