10 ms·

The article and video trumpet the curiosity-based approach: you tell the robot to be "curious" about the door handle, and give a reward when the task of passing

The article and video trumpet the curiosity-based approach: you tell the robot to be "curious" about the door handle, and give a reward when the task of passing through the door is complete, and magic happens. I'm comfortable with my lack of understanding of the magic. I'm embarrassed by my lack of understanding of this "curiosity" concept. Does anyone have different words than those used in the article which might unlock my comprehension?

solardev 3y ago
The article links to another article talking about that, in case you missed it: https://towardsdatascience.com/curiosity-driven-learning-made-easy-part-i-d3e5a2263359 https://towardsdatascience.com/curiosity-driven-learning-mad...
- stevenae 3y ago
  I forsee a really cool hacking movie being made based on this article.
calamari4065 3y ago
Say you have a bunch of AI agents playing a game where they need to jump up on a platform and push a button. You could give them no information and let them randomly move around until one happens to find the button. But what if you just told them there was "something" interesting in a particular spot? You could skip hours and hours of training them just to find the button by just giving them a hint. Think about the way a baby learns. They spot an object that catches their interest, and do whatever they can to get to it and inspect it. The difference is that the robot doesn't have an inherent mechanism for identifying interesting objects. You have to give it a hint as to what it should consider interesting, and it can figure the rest out from there.
deepnet 3y ago
Curiosity (in this context) is a Loss function based on the error between the observed next state and agent's prediction of the next state given the current state and an action. The states are encoded in a feature space that ignores irrellivancies that don't effect and are out of the agent's control ( e.g. leaves blowing on trees ) Curiosity provides a method to learn this feature space from the raw pixel input via inverse dynamics. The feature space is compact, relevant, sufficient, & stable which is ideal for RL. This Curiosity Loss function entices the agent to explore states it has not been in and areas with complex dynamics. These are states with high uncertainty which the agent reduces by becoming familiar with them. This is a self-supervised method with no extrinsic human designed reward. The agent aquires general knowledge about its enviroment and it is analagous to being curious.

solardev 3y ago

The article links to another article talking about that, in case you missed it: https://towardsdatascience.com/curiosity-driven-learning-made-easy-part-i-d3e5a2263359 https://towardsdatascience.com/curiosity-driven-learning-mad...

stevenae 3y ago
I forsee a really cool hacking movie being made based on this article.

calamari4065 3y ago

Say you have a bunch of AI agents playing a game where they need to jump up on a platform and push a button. You could give them no information and let them randomly move around until one happens to find the button. But what if you just told them there was "something" interesting in a particular spot? You could skip hours and hours of training them just to find the button by just giving them a hint. Think about the way a baby learns. They spot an object that catches their interest, and do whatever they can to get to it and inspect it. The difference is that the robot doesn't have an inherent mechanism for identifying interesting objects. You have to give it a hint as to what it should consider interesting, and it can figure the rest out from there.

deepnet 3y ago

Curiosity (in this context) is a Loss function based on the error between the observed next state and agent's prediction of the next state given the current state and an action. The states are encoded in a feature space that ignores irrellivancies that don't effect and are out of the agent's control ( e.g. leaves blowing on trees ) Curiosity provides a method to learn this feature space from the raw pixel input via inverse dynamics. The feature space is compact, relevant, sufficient, & stable which is ideal for RL. This Curiosity Loss function entices the agent to explore states it has not been in and areas with complex dynamics. These are states with high uncertainty which the agent reduces by becoming familiar with them. This is a self-supervised method with no extrinsic human designed reward. The agent aquires general knowledge about its enviroment and it is analagous to being curious.