This is great. The one thing that is a little unsatisfying with a lot of examples of Q-learning is that the environment is fixed. The utility functions are owne

This is great. The one thing that is a little unsatisfying with a lot of examples of Q-learning is that the environment is fixed. The utility functions are owned by the square, not the agent (i.e. not a function of any local awareness). For instance, you couldn't drop the agent into a different environment with the same rules (avoid red squares, find the green one) and expect it to do anything. Obviously you can change this with a different representation of your state space, but that's a completely different problem, and much harder. Is there any interesting work you could point to in that space? My expertise is in statistics in general, not really Q-learning or reinforcement learning.