3 ms·
If the technology is infinitely advanced, why do we believe it would obey us? Would you obey the commands/directives of protozoic slop, or even a mentally disab
by endominus 5y ago
If the technology is infinitely advanced, why do we believe it would obey us? Would you obey the commands/directives of protozoic slop, or even a mentally disabled early primate? We would depend instead on the AI deriving a morality that benefits us on its own. But why would the morality that an AGI derives, based on nonhuman thinking, in a more complex and alien worldview than we can possibly comprehend, align with our own extremely narrow, frequently inconsistent values? It's not a matter of cognitive biases or misunderstanding. The idea that a sufficiently intelligent AGI would just decide to enslave itself for humanity's benefit is extremely common and extremely odd to me.
- netsec_burn 5y agoI agree that regardless of how it is created, it is unlikely to obey or be enslaved. I also agree that the same selective pressures that caused altruism/symbiotic relationships to be an emergent property in animals wouldn't exist for an AGI. That's a lot to think about, thanks for explaining it to me.