Curiosity (in this context) is a Loss function based on the error between the observed next state and agent's prediction of the next state given the current sta

Curiosity (in this context) is a Loss function based on the error between the observed next state and agent's prediction of the next state given the current state and an action. The states are encoded in a feature space that ignores irrellivancies that don't effect and are out of the agent's control ( e.g. leaves blowing on trees ) Curiosity provides a method to learn this feature space from the raw pixel input via inverse dynamics. The feature space is compact, relevant, sufficient, & stable which is ideal for RL. This Curiosity Loss function entices the agent to explore states it has not been in and areas with complex dynamics. These are states with high uncertainty which the agent reduces by becoming familiar with them. This is a self-supervised method with no extrinsic human designed reward. The agent aquires general knowledge about its enviroment and it is analagous to being curious.