3 ms·
But the reward (i.e., meaning) is defined by a human engineer. The point of the article is to say that we don’t have AGI yet because today’s AI needs humans to
by madmax96 4y ago
But the reward (i.e., meaning) is defined by a human engineer. The point of the article is to say that we don’t have AGI yet because today’s AI needs humans to ascribe meaning to its decisions.
- ftio 4y agoSerious-ish question: how do we know the ‘meaning’ we perceive doesn’t also originate from some external agent?
- pulvinar 4y agoFor base desires, that 'agent' is evolution. E.g., we can't will ourselves to stop breathing for the next hour, no matter what our higher-level desire might be. Many of our higher desires are set for us (taught) by other humans: our parents and society. We wouldn't want an AGI that was able to set all of its own desires, any more than we'd want that in our children. Rather dangerous indeed.
- dsubburam 4y agoAgree. Instead of "meaning" I might just use "desire". The AI is trained to meet the human engineer's chosen desire. The engineer specifies this desire through a loss function (supervised learning) or rewards (reinforcement learning). Two more points: 1. This "desire" applies only to the training algorithm. The deployed AI (running in inference) is no longer trying to minimize a loss function or maximize discounted rewards. 2. This "desire" is not a "deliberate desire". It is more like an "animal desire" such as wanting to eat because of hunger. Aristotle draws this distinction. Such a desire is automatic (rather than deliberate) and is not part of choice-making.