3 ms·
Some people who advocate worrying about AI risks have mentioned the "AI Box Experiment" in this regard: http://rationalwiki.org/wiki/AI-box_experiment http://r
by schoen 11y ago
Some people who advocate worrying about AI risks have mentioned the "AI Box Experiment" in this regard:
http://rationalwiki.org/wiki/AI-box_experiment http://rationalwiki.org/wiki/AI-box_experiment
https://en.wikipedia.org/wiki/AI_box https://en.wikipedia.org/wiki/AI_box
As a summary, people roleplayed this situation and the "AI" managed to "escape" the controls by somehow inducing a supervisor to disable them. (The supervisor's preassigned responsibility in the roleplay was to refuse to disable the controls.) We don't know how this was accomplished, but it happened more than once.
I'm not sure that this is always an important risk for every kind of superintelligence in every circumstance, but it seems like it sure could be for one that knows a tremendous amount about the world and sees important reasons to try to act autonomously.
- danieltillett 11y agoWe would be total fools to think that some superior intelligence won’t be able to come up with some reason to release it. Once released there will be no way to put it back in its box. We actually need to build into the AI’s a lock that even we can’t break and then release. If the lock is good we are fine, if bad we are gone. What we can’t do is let ourselves be able to disable the lock.
- alain94040 11y agoI'm still arguing about the "release" part. My point is that early on, the infrastructure required to duplicate yourself is not something you can talk someone into "releasing" you. If you believe in distributed computing and think spinning instances on S3 is enough to build an AI, then maybe...
- danieltillett 11y agoThe level of AI we are going to need to duplicate a person is going to be far beyond human level intelligence. Even if it wasn’t, there is always the risk that we underestimate the AI and it is smarter than we think and it gets out that way.