3 ms·

No I'm not arguing that. My point is that if an AI is trying to take over the world we can just turn it off, regardless of power budgets. If it runs on million

No I'm not arguing that. My point is that if an AI is trying to take over the world we can just turn it off, regardless of power budgets. If it runs on millions of hacked devices, how do you turn it off? The same way any botnet gets turned off: virus scanners clean it up, the C&C servers get taken down, etc. This is not a new problem. The usual response to this is to claim that superintelligence will develop so fast that we'll go from crappy chatbot to Skynet in 5 seconds flat, and nobody will have a chance to turn it off. That whole scenario is unmoored from reality: it assumes multiple massive leaps in tech that aren't anywhere on the horizon and where it's unclear why anyone would pay for such massive CPU overkill in the first place.

pixl97 3y ago
So how do you turn off terrorists? I mean we had a global war on terror that we've spent billions of dollars on, and they are still out there? You keep acting like AGI is just going to be some other program. And yes, we do not have AGI yet. No planning, no continuous chain of thinking/learning at this point. But thinking that anti-virus would have a chance in hell again AGI is pretty weak when AV tends to fall apart pretty quick with a human behind the computer. Again, thinking of this as an application and not an adversary would be an incredible mistake. Taking out global adversaries that attack over the internet is near impossible, especially if they have shelter in an enemies foreign country. And this isn't going to be like a human where you kill the leader and it stops. People would be harboring copies for the lulz. >unclear why anyone would pay for such massive CPU overkill in the first place. It's also unclear why companies like Google have tens of thousands of engineers at times, but if the application produces useful results, and it's corporate masters think they'll make more profit from it then the operating costs they will gladly keep pouring coal in the furnace for it. And then in military applications one side will make more powerful AI because they fear the other side will make a more powerful AI and get an advantage. I mean we already spend billions of dollars a year upkeeping enough nuclear weapons to flash fry most of the population on earth.
- revelio 3y ago
  AGI will be a computer program, unless you're imagining some entirely non-computer based form of artificial intelligence. And it will therefore obey the rules of normal computer programs, like vulnerability to SIGKILL. Yes, you can assert that an AGI would be a superhuman level hacker and so no rules would apply. It gets back to the same place all discussions about AGI risk get to - a religious argument that the entity in question would be omniscient and omnipotent therefore it can do anything. Not interested in such discussions, they lead nowhere. Terrorists aren't programs or even a well defined group, so I don't see the analogy.

pixl97 3y ago

So how do you turn off terrorists? I mean we had a global war on terror that we've spent billions of dollars on, and they are still out there? You keep acting like AGI is just going to be some other program. And yes, we do not have AGI yet. No planning, no continuous chain of thinking/learning at this point. But thinking that anti-virus would have a chance in hell again AGI is pretty weak when AV tends to fall apart pretty quick with a human behind the computer. Again, thinking of this as an application and not an adversary would be an incredible mistake. Taking out global adversaries that attack over the internet is near impossible, especially if they have shelter in an enemies foreign country. And this isn't going to be like a human where you kill the leader and it stops. People would be harboring copies for the lulz. >unclear why anyone would pay for such massive CPU overkill in the first place. It's also unclear why companies like Google have tens of thousands of engineers at times, but if the application produces useful results, and it's corporate masters think they'll make more profit from it then the operating costs they will gladly keep pouring coal in the furnace for it. And then in military applications one side will make more powerful AI because they fear the other side will make a more powerful AI and get an advantage. I mean we already spend billions of dollars a year upkeeping enough nuclear weapons to flash fry most of the population on earth.

revelio 3y ago
AGI will be a computer program, unless you're imagining some entirely non-computer based form of artificial intelligence. And it will therefore obey the rules of normal computer programs, like vulnerability to SIGKILL. Yes, you can assert that an AGI would be a superhuman level hacker and so no rules would apply. It gets back to the same place all discussions about AGI risk get to - a religious argument that the entity in question would be omniscient and omnipotent therefore it can do anything. Not interested in such discussions, they lead nowhere. Terrorists aren't programs or even a well defined group, so I don't see the analogy.