4 ms·

So lets say I take the trained network, and flip the color of the pixels, and make some other cosmetic changes (keeping the game intact). Will the network then

• gwern 10y ago
  The ALE agents are almost always using grey-scale pixels in the first place (to save on processing and NN space requirements), so flipping the color makes no difference.
  • seanwilson 10y ago
    Is it learning the specific level layout or can it adapt when the screens are different and order swapped?
    • gwern 10y ago
      DQN is stateless, so at every point it's reacting to just the current frame (technically, the average of the last 4 frames IIRC) being fed into a CNN and outputing a motion. It doesn't care about ordering all that much because it has no memory. In this case, it's probably learning stuff like 'if there's a key, go towards it' and 'if you have a key, go towards a barrier'. If each screen keeps working the same way, then I guess it would achieve about the same performance as it does now. New screens with new layouts/patterns of enemies will test how well its learned heuristics generalize, though. Personally, I think that someone should be trying a DQN with an RNN rather than CNN in it to see if that helps on the harder levels. Or better yet, combine it with some of the memory mechanisms and see if it can start doing some real long-term planning.
      • SandB0x 10y ago
        I believe the input state is the past 4 frames stacked or concatenated.
• drcode 10y ago
  I think in general these types of neural networks are flexible enough to be resistant to trivial changes like this- Whether that specific change for this specific NN would affect results would likely need to be checked empirically.