Definitely use a GPU if you can - it's just so much faster. You could also try training it on a powerful machine in the cloud if you don't mind spending the mon

Definitely use a GPU if you can - it's just so much faster. You could also try training it on a powerful machine in the cloud if you don't mind spending the money for it... I'm running a 8 core machine, 12+ GB RAM, and a 980 GPU. It takes roughly 24 hours for 1 million steps in Mario...