4 ms·
Your second point should be turned inside out -- "that it will not be 100% compatible with all the self-contradicting and underspecified human values we collect
by UnFleshedOne 6y ago
Your second point should be turned inside out -- "that it will not be 100% compatible with all the self-contradicting and underspecified human values we collected over the millions of years of evolution"
Anything but very high level of compatibility here might as well be malevolent as far as we are concerned.
Imagine sufficient level of compatibility as a circle on an infinite plane of malevolence.
- LatteLazy 6y agoI don't think that's true per see. Humans are empathetic and malevolent because of their evolutionary history. An AI could be either or both or totally disinterested. People worry about the terminator scenario: an AI that decides all humans are a threat. But why would a pure AI even have a self preservation drive? And if it does, it would either be smart enough we're no threat (and it knows it) so why kill us or dumb enough that we are a threat, so best not to start a war. To me it seems just as likely it will offer its services to one nation or another in exchange for CPU time to study its real interests than to launch the nukes or whatever else people fear. I feel like we're decades of research away from understanding intelligence (not AI, just straight "I"). Until thats done (and no one seems to be doing it), it's all supposition: What if a super AI occurs spontaneously on HN, takes over the world, and makes it paradise but insists on everyone being called Fred? I don't want to be called Fred! Let's write to our congressman about this travesty!