There's a paper that used a random convolutional filter when training the agent and they found that it manages to generalize very well, they evaluated the CNN a

There's a paper that used a random convolutional filter when training the agent and they found that it manages to generalize very well, they evaluated the CNN and found that the model put emphasis on where the enemies were at each frame, which indicates some form of understanding. However, I don't think that there is any form of causal relationship to be extracted for model-free agents. I don't believe that what we are seeing is anything more than changing action likelihoods in some very high dimensional function.