2 ms·
"We can make an AI with desires. For example, we can give it a reward every time you push a button." I wouldn't call that a "desire" in the fashion we've discu
by joe_the_user 11y ago
"We can make an AI with desires. For example, we can give it a reward every time you push a button."
I wouldn't call that a "desire" in the fashion we've discussed it. I guess it comes down to fundamental disagreement over how GAIs could be created. I think it's obvious that the creation of a GAI would require a very careful engineering of "understanding" in a broad sense - "knowing what people mean" "caring about X" etc. in order to occur at all.
If just complexity and rewards are enough to create a thing that intentionally increases resources and makes longer term plans, then it seems like we are indeed in trouble. But I think that's implausible.
- Houshalter 11y agoCall it a desire/goal/value/utility function/whatever. It's a thing which the AI wants and creates plans to obtain. This isn't that different than how animals and humans work, as far as we know. We get pleasure from different things, and our brains seek actions which lead to those things. Creating an AI is only a matter of coming up with really good prediction algorithms. Algorithms that predict the future reward the AI will get from an action. While this is of course a difficult problem, it's not impossible, and we are making a ton of progress on it. Other models of AI all have similar problems though. At some level you need to program the AI's goals explicitly. Even if that goal is "try to understand what we mean, and do that", you still have to figure out a way of writing that down in code. It's an impossible task.
- joe_the_user 11y agoOK, I think I understand where you coming from. While I don't think a GAI could arise from your scenario - just loops - I can see now that you are making a plausible argument and given that the whole field is extremely uncertain, it is reasonable to be worried by a plausible, problematic scenario.