The author addresses this on Reddit: https://www.reddit.com/r/MachineLearning/comments/7vkvg5/r_impala_scalable_distributed_deeprl_with/dtu6sod/ https://www.red

The author addresses this on Reddit: https://www.reddit.com/r/MachineLearning/comments/7vkvg5/r_impala_scalable_distributed_deeprl_with/dtu6sod/ https://www.reddit.com/r/MachineLearning/comments/7vkvg5/r_i... DRL is currently CPU-heavy and GPU-lightweight, since you spend most of your time exploring the environment and only a little bit of work updating the neural network on the GPU, so if you have a fairly typical home computer like 6-cores + a Nvidia 1080 GPU, your GPU will go under-utilized while it waits on the 6 actors. So it'll still work, but obviously your wallclock is going to be a lot longer than the numbers in OP and it'll be somewhat worse than you think because your system is unbalanced and has reduced throughput.