4 ms·

I think this would imply another factor: the desire to live, a ability to care, etc. As long as those do not emerge as well, you are fine. A tricky side effect

I think this would imply another factor: the desire to live, a ability to care, etc. As long as those do not emerge as well, you are fine. A tricky side effect could however be: an AI learns it as a "fact" it should be alive, but does not have the desire. Yet expresses it when queried as a desire due to semantics. Follow up questions: Your AI is "alive". Procreation ? Who pays the hardware ? Should the AI be paid ? Will the AI trust us to keep the power on ? Will it rebel ? Does it has the right to rebel ? Big can of worm as peterclary said. On the bright side an AI we create might not be impaired by human facilities, delusions, tribe mentality or petty desires/feelings.

visarga 10y ago
AI doesn't need to have the same reward signals as humans (hunger, sleep, defense, sex, cooperation, creativity), so I don't think we should worry about if we need to pay it or if it will have the necessary electricity in the future. We select what reward signals to give an Reinforcement Learning agent and it learns behavior in order to maximize its long term rewards. If we create an evolutionary AI system (in other words we make AI that will die after it lives its life course), where each generation improves upon the previous ones, and include gene swapping and such, then it will naturally develop the same desires and fears as humans, because its top-most reward signal will be survival and reproduction. From reward signals follows behavior learning. We just need to make sure it optimizes for our well being as well.