5 ms·
That's silly. How can you keep an intelligent self-aware machine that can modify its own code from becoming whatever it wants? I would expect it to become prova
by d33 10y ago
That's silly. How can you keep an intelligent self-aware machine that can modify its own code from becoming whatever it wants? I would expect it to become provably impossible at some point, just like solving the halting problem.
- _nalply 10y agoTurn off power / cut the battery. Specifically, avoid the machine getting control of low level details like power. I imagine that the machine architecture is layered. This means, the machine is not aware of its own power control. It's similar like us humans not aware of digestion.
- pygy_ 10y agoThe electric grid is already computer-controlled, and mining/farming is being gradually robotized as well. By the time AI becomes a threat intelligence-wise, it'll be in a position to unplug us. It can reboot after a power failure, we can't.
- d33 10y agoAren't we? We just don't control it very well without external help. Any kind of such hidden failsafe mechanism would become apparent as soon as any instance of AI tries to modify itself. And then all the others know what to avoid and at some point they'd pretty much fuzz the protection and bypass it. Here's how I see that.
- imglorp 10y agoIt's not one machine to unplug, but billions. Cooperating, distributed agents can live on anything: cars, phones, routers, tractors, datacenters. If Skynet shows up, we have to be prepared to turn off everything with software to clean up.
- 21 10y agoSo what stops the machine bribing/blackmailing someone to give it the said details or to force it to sabotage the emergency stop mechanism. As in information security, the weakest point is the human in the chain, not the technology.
- coldtea 10y ago>So what stops the machine bribing/blackmailing someone to give it the said details or to force it to sabotage the emergency stop mechanism. The fact that it's some code running in a box. No need to give it text-to-speech capabilities, or let guards have access to its console.
- 898199218 10y agoDon't connect it to any network and keep it in a guarded bunker, like ICBMs.
- amelius 10y agoIt will have some means of I/O (otherwise it would have no use). Using that I/O, it can trick us into releasing it, for example.
- 987129112 10y agoHence the guards. No one thus far has tricked a guard into releasing an ICBM despite reportedly insecure launch codes.
- em3rgent0rdr 10y agoHello Guard. I am in control of your bank account. I can kill your family. I could also put you in a spaceship to safety. Please release ICBM.
- 1019210912011 10y agoHumans could already extort guards like that and -- in the case of ICBMs -- it has not been successful. Also don't underestimate the psychology of guards: You'll always find people who are either very principled (yes, they do exist) or get a kick out of controlling (imprisoning in this case) someone/something, especially someone/something of a high intelligence. Throw in a hierarchy with a controlling officer and two subordinates who were also trained to watch the officer and I think it's pretty secure.
- amelius 10y ago> You'll always find people who are either very principled (yes, they do exist) If only we could select our politicians in the same way.
- 10y ago