3 ms·
Don't develop human-like minds, or prove the stability of a goal system under recursive self-improvement. (If you offer Gandhi a pill that he knows will make hi
by Jach 16y ago
Don't develop human-like minds, or prove the stability of a goal system under recursive self-improvement. (If you offer Gandhi a pill that he knows will make him want to kill people, he won't take it, because he doesn't want to want to kill people.) If you throw a bunch of crap together and somehow get a mind from it, it's most likely not going to be anything resembling human (and so you'd be right that not all AIs would reach the same conclusions). I'm not an AI researcher so I'll stop here, but you're assuming that many AGIs are going to even have a concept of "taste" or "dissent" let alone a preference for it.
- jerf 16y agoActually, that's why I carefully stuck to phrasing it in evolutionary terms. I have no idea what AIs may or may not arise, but if the situation has spun this far out of human control regardless of how there may very well be a "species" of robot that desires to eat humans, it won't last long against its robotic competition. Plus the other robotic competition probably eliminated the humans, anyhow.