You could also use deep features (pre-trained for ImageNet classification) and use them in your Q-function approximator in such a way that the Q-function is lin

You could also use deep features (pre-trained for ImageNet classification) and use them in your Q-function approximator in such a way that the Q-function is linear wrt some high-level features. Then, you get the best of both: being able to process complex visual input while being able to do reinforcement learning with very few training trajectories. See [1] for an example (in simulation). [1] http://rll.berkeley.edu/visual_servoing/ http://rll.berkeley.edu/visual_servoing/