3 ms·
From the article: lots of behaviors that are necessary to advance within the game do not help increase the score until much later I found this passage very int
by ineedasername 8y ago
From the article: lots of behaviors that are necessary to advance within the game do not help increase the score until much later
I found this passage very interesting, as it seems like the definition of the benefits that can extend from delayed gratification. That optimizing for a local maxima isn't always the best way to get the best global maxima. Maybe algorithms that can encapsulate that will be useful in getting corporations away from the habit of chasing the highest short-term gains at the expense of long term viability.
- jetrink 8y agoAnother way to look at it is that these games have less obvious causal relationships. If the game is Pac-Man, it is immediately obvious that eating a dot makes the score go up, because the effect always and immediately follows the cause. The further the effect is separated from the cause, the more possibilities the machine must entertain. Imagine, in a different game, you activate three switches and a door opens. Why did the door open? Maybe all three switches must be activated for the door to open. Maybe the final switch controls the door and the other switches control something else. Maybe you must always activate the switches in a particular order as a security measure. The game probably gives context (e.g. the third switch is labeled 'open door') that a human can use to eliminate the many possibilities, but the machine must experiment before knowing what is relevant. When you separate cause and effect in time, the machine must deal with many possible 'switches'.