3 ms·
I am not super-intelligent, but I myself can imagine lots of ways that a super intelligence might outsmart an off switch. One being to hide what it is doing unt
by jjeaff 3y ago
I am not super-intelligent, but I myself can imagine lots of ways that a super intelligence might outsmart an off switch. One being to hide what it is doing until it is able to write and distribute a virus to the outside world that distributes itself. Or perhaps it could convince some other humans to physically take over and stop anyone from flipping the off switch.
The question you should ask yourself is how much damage could the world's most skilled hacker do if he could clone his knowledge to a billion hackers all working together in close synchronization and thinking thousands of times faster than any human, ever.
And we have already seen that due to the black box nature of most modern AI implementations, it is not so easy to put rules in place. People are frequently finding "jailbreaks" to get around the restrictions.
- dkjaudyeqooe 3y agoWhy haven't "hackers" hacked into nuclear weapons infrastructure? The street cred would be incredible! I think it's possible to build secure anything if that is the goal. Most software is written by people motivated to write something useful and/or neat, or collect a paycheck, and don't focus on security.
- bumby 3y agoI’m pretty sure Stuxnet did hack it’s way into nuclear weapons infrastructure.
- kortilla 3y agoA refinement facility for nuclear material isn’t quite the same thing
- jjeaff 3y agoIt's definitely part of the infrastructure of nuclear weapons. That's why it was hacked in the first place. They were making weapons grade nuclear material.
- kortilla 3y agoSo is a steel plant though. Keeping a missile from being hacked is much different than something in the supply chain.
- bumby 3y agoConsidering the number of steel plants and the relative cost of the material, that’s not a very good comparison. The whole point of Stuxnet was that the facility was a critical single failure point in the supply chain. When it went down, it disrupted the project for years. I don’t think the same dynamic holds for steel manufacturing. If you think they’re comparable, try getting into Oak Ridge or similar; I suspect you’ll find it very well guarded compared to a steel plant.
- jjeaff 3y agoBecause they aren't skilled enough or don't have the resources. But do you really think that means no super intelligence would be able to?
- woopsn 3y agoHow much damage could one do with ~20 terawatts of brain power? A lot. But nothing clandestine.