4 ms·
Obviously it's impossible to prove definitive statements about every possible potential action, as as per the halting problem some of those actions are unprovab
by Veedrac 6y ago
Obviously it's impossible to prove definitive statements about every possible potential action, as as per the halting problem some of those actions are unprovable.
It is as ridiculous to suggest that this means you can't contain a superintelligence as it is to suggest it means you can't, I don't know, go buy bread. In both cases you could analyze running a program that doesn't halt but you can't prove doesn't halt, and lock up your reasoning algorithm. The sensible thing is to not do that.