6 ms·
Usually in these discussions we use the term "Unfriendly AI" instead of "Hostile AI". That gets at an important distinction: these AIs don't want humanity to d
by saucetenuto 13y ago
Usually in these discussions we use the term "Unfriendly AI" instead of "Hostile AI". That gets at an important distinction: these AIs don't want humanity to die, it's just that they don't want humanity to live and they're presumed to have the ability to steal all our resources for themselves. Humanity still dies, but only incidentally. The author discusses this point a little but I think it's important enough to put it front and center, in our terminology.
It's interesting to think of corporations as being Unfriendly in this sense. The analogy isn't perfect, though: humans make up a corporation's computing substrate, so they're forced to value humans more than the classic Unfriendly AI would.
- dnautics 13y agois there a third category of AI which doesn't necessarily care to live or not, but winds up destroying humanity because of its Lennie-like power (possibly even in the service of humanity)?
- saucetenuto 13y agoMaybe? I'm not sure what you mean.
- eurleif 13y agoI took it to mean http://wiki.lesswrong.com/wiki/Paperclip_maximizer http://wiki.lesswrong.com/wiki/Paperclip_maximizer
- maaku 13y agoYes, "Unfriendly AI", as in not-provably-friendly AI, which is not the same thing as "Hostile AI".