3 ms·
Reinforcement learning has a pretty useful framework for thinking about this called the exploration-exploitation dilemma. In order for an agent to fully exploi
by briga 8y ago
Reinforcement learning has a pretty useful framework for thinking about this called the exploration-exploitation dilemma.
In order for an agent to fully exploit it's environment it first has to explore that environment, and this is where curiosity becomes useful. In fact, last year there was a paper that tried to introduce curiosity to RL-based learning models [1]
I suspect some sort of cross-pollination of these two areas of research would be useful.
[1] https://www.google.com/url?q=https://arxiv.org/pdf/1705.05363&sa=U&ved=2ahUKEwiLwb7Kr_DeAhUE7Z8KHaTjDkQQFjAAegQIAhAB&usg=AOvVaw1tEAL8VpFHjtFkDIhSK_RU https://www.google.com/url?q=https://arxiv.org/pdf/1705.0536...
- deepnotderp 8y agoI view intrinsic curiosity as different from exploitation-exploration, the latter of which has its roots as far back as WWII and bandit problems. Imho, curiosity is something fundamentally different than randomly trying new stuff in hopes of finding something with higher reward. Some of the artificial curiosity/intrinsic motivation work in RL seems to suggest this as well.
- winchling 8y ago>Imho, curiosity is something fundamentally different than So what's it all about then? Go on, I'm curious...
- deepnotderp 8y agoIf I knew then I would be in a pretty powerful position :) But at least some starting point: exploration-exploitation is basically chasing something with a better outcome, for example, if I'm working on some project, I might look at random websites for hopes of finding something useful for that project, on the other hand, curiosity would be me just looking at things that are "interesting". Right now in RL artificial curiosity research we use novelty as a heuristic for "interesting" but we haven't really solved what "interesting" is yet I think.
- winchling 8y agoRight! Curiosity could very well be about optimising the structure of one's existing knowledge rather than finding out new things per se. A distinctly non-random process.
- tim44 8y agoThis is the first I've heard about this topic, but what's different about intrinsic curiosity to me is that its a record, theres some of yourself in there, you are testing your own experiences against the world from a beginning point of recalled experience in the moment, rather than I guess exploitation-exploration is throwing things at the wall? But I'd guess that still comes from a place of intrinsic curiosity even if we can't see it.
- elisee 8y agoSee also this recent blog post + paper by OpenAI, they designed curiosity in an agent by having it test how well it can predict the output of a randomly initialized neural network from the game's current frame + input. The better the prediction, the more its mental model of the world is correct, and the less it gets rewarded (encouraging it to go and find unexplored situations). https://blog.openai.com/reinforcement-learning-with-prediction-based-rewards/ https://blog.openai.com/reinforcement-learning-with-predicti...