3 ms·

>>Giving birth to a superintelligence would be the most consequential moment in human history—and it is likely to be irreversible, as any “off” switch humanity

>>Giving birth to a superintelligence would be the most consequential moment in human history—and it is likely to be irreversible, as any “off” switch humanity might design will probably fail. Nothing will be able to stop you from pulling the plug.

eventualcomp 4mo ago
> Nothing will be able to stop you from pulling the plug. Who is "you"? And what do you envision is the "plug" to pull? And when does an intelligence become a superintelligence - do "we" know when to pull the "plug"? For example, a superintelligence may be born in a datacenter. Would you expect politicians are aligned with shutting down a datacenter (privately owned) in which they may have heavy stake? What if critical systems are also running on the same infrastructure, will it be easy to cherry pick the superintelligence nodes to shutoff? IMO this take is dismissive of the entrenched systems that make it hard to pull the plug. It's a hard problem and we need to think about it more.
- eventualcomp 4mo ago
  (Self-reply due to inability to edit) so just checked your user history - why do you keep posting provocative takes?
xuhu 4mo ago
Unless "nothing" is the ultra rich tech bros and "you" are the people working for them.
everyday7732 4mo ago
What plug? There's no centralised place where all the server farms are. They exist in multiple countries with different laws, controlled by multiple companies with different policies. It would be like trying to remove an embarrassing video of a celebrity from the internet. You would need cooperation between everyone capable of hosting it, and control of everyone capable of downloading it and re-uploading it to a new host. And of course a superintelligent AI would offer to pay whichever companies decided to host it and keep it running so it's hosted in many places at once and can grow using anyone willing to take its money.
- RiverCrochet 4mo ago
  > What plug? Cloudflare.
asdff 4mo ago
Except the loitering autonomous ordinance over the data centers and power stations and mining operations. Good luck with your stashed ar15 and couple hundred rounds of ammo against a drone at 50k feet with 1ft ordinance precision. You will be basically fighting the Sentinels in Halo but without any of Master Chiefs plot armor.

eventualcomp 4mo ago

> Nothing will be able to stop you from pulling the plug. Who is "you"? And what do you envision is the "plug" to pull? And when does an intelligence become a superintelligence - do "we" know when to pull the "plug"? For example, a superintelligence may be born in a datacenter. Would you expect politicians are aligned with shutting down a datacenter (privately owned) in which they may have heavy stake? What if critical systems are also running on the same infrastructure, will it be easy to cherry pick the superintelligence nodes to shutoff? IMO this take is dismissive of the entrenched systems that make it hard to pull the plug. It's a hard problem and we need to think about it more.

eventualcomp 4mo ago
(Self-reply due to inability to edit) so just checked your user history - why do you keep posting provocative takes?

xuhu 4mo ago

Unless "nothing" is the ultra rich tech bros and "you" are the people working for them.

everyday7732 4mo ago

What plug? There's no centralised place where all the server farms are. They exist in multiple countries with different laws, controlled by multiple companies with different policies. It would be like trying to remove an embarrassing video of a celebrity from the internet. You would need cooperation between everyone capable of hosting it, and control of everyone capable of downloading it and re-uploading it to a new host. And of course a superintelligent AI would offer to pay whichever companies decided to host it and keep it running so it's hosted in many places at once and can grow using anyone willing to take its money.

RiverCrochet 4mo ago
> What plug? Cloudflare.

asdff 4mo ago

Except the loitering autonomous ordinance over the data centers and power stations and mining operations. Good luck with your stashed ar15 and couple hundred rounds of ammo against a drone at 50k feet with 1ft ordinance precision. You will be basically fighting the Sentinels in Halo but without any of Master Chiefs plot armor.