> For example, imagine a television displaying nothing but static on its screen. Such a thing would quickly engage the curiosity of a purely novelty-seeking age

> For example, imagine a television displaying nothing but static on its screen. Such a thing would quickly engage the curiosity of a purely novelty-seeking agent, because a square of randomly flickering visual noise is, by definition, totally unpredictable from one moment to the next. Since every pattern of static appears entirely novel to the agent, its intrinsic reward function will ensure that it can never cease paying attention to this single, useless feature of the environment — and it becomes trapped. Reminds me of http://people.idsia.ch/~juergen/interest.html http://people.idsia.ch/~juergen/interest.html > To discourage the controller from focusing on truly unpredictable, random inputs (such as uninteresting details of white noise), later approaches model the expected progress of the predictor: parts of the world where the predictor fails to learn (no data compression progress!) become less interesting than those where its predictions improve. This seems like a better/more general solution than the one presented in the article, since the agent could otherwise get stuck with noise over which it does have causal influence. The agent could even generate, using its actuators, noise that is unpredictable and chaotic, but not interesting.