4 ms·

From the google one: > The environment also contains a TV for which the agent has the remote control. There is a limited number of channels (each with a distin

From the google one: > The environment also contains a TV for which the agent has the remote control. There is a limited number of channels (each with a distinct show)... even if the order of shows appearing on the screen is random and unpredictable, all those shows are already in memory! So in google's "Curiosity and Procrastination in Reinforcement Learning" they could not handle a TV with pure noise (snow) as it could not remember all that noise.

Jabbles 8y ago
I don't think that's a reasonable conclusion to draw from the fact the TV wasn't pure noise. Why would the agent not be able to determine that every noisy frame was one step away from every other?
- mooneater 8y ago
  The google paper uses memory. You cant remember a never ending set of 2D random noise, there are limitless variations.
  - Jabbles 8y ago
    They don't remember all the previous scenes, just ones which are "novel" enough. But how do we decide whether the agent is seeing the same thing as an existing memory? Checking for an exact match could be meaningless: in a realistic environment, the agent rarely sees exactly the same thing twice. For example, even if the agent returned to exactly the same room, it would still see this room under a different angle compared to its memories. Instead of checking for an exact match in memory, we use a deep neural network that is trained to measure how similar two experiences are.
    - mooneater 8y ago
      Ok. Though it still depends on limited number of tv shows. If there is an unlimited number of shows and the agent walks right up to it, I think its still trapped.
      - jhurliman 8y ago
        Aren’t we all. http://rickandmorty.wikia.com/wiki/Interdimensional_Cable http://rickandmorty.wikia.com/wiki/Interdimensional_Cable
deep_etcetera 8y ago
This approach will be thwarted if the time is shown in the observation space. If this is so then every kth state will give a novelty reward and every other state will give zero, it doesn't matter what the agent does.

Jabbles 8y ago

I don't think that's a reasonable conclusion to draw from the fact the TV wasn't pure noise. Why would the agent not be able to determine that every noisy frame was one step away from every other?

mooneater 8y ago
The google paper uses memory. You cant remember a never ending set of 2D random noise, there are limitless variations.
- Jabbles 8y ago
  They don't remember all the previous scenes, just ones which are "novel" enough. But how do we decide whether the agent is seeing the same thing as an existing memory? Checking for an exact match could be meaningless: in a realistic environment, the agent rarely sees exactly the same thing twice. For example, even if the agent returned to exactly the same room, it would still see this room under a different angle compared to its memories. Instead of checking for an exact match in memory, we use a deep neural network that is trained to measure how similar two experiences are.
  - mooneater 8y ago
    Ok. Though it still depends on limited number of tv shows. If there is an unlimited number of shows and the agent walks right up to it, I think its still trapped.
    - jhurliman 8y ago
      Aren’t we all. http://rickandmorty.wikia.com/wiki/Interdimensional_Cable http://rickandmorty.wikia.com/wiki/Interdimensional_Cable

deep_etcetera 8y ago

This approach will be thwarted if the time is shown in the observation space. If this is so then every kth state will give a novelty reward and every other state will give zero, it doesn't matter what the agent does.