5 ms·
People keep saying it's "not feasible" to do deep learning on a CPU. Is that actually true? I'm thinking of papers like this one[0] from Uber. that make the cla
by tree_of_item 8y ago
People keep saying it's "not feasible" to do deep learning on a CPU. Is that actually true? I'm thinking of papers like this one[0] from Uber. that make the claim that CPU training is very much feasible, in this case with a cluster of 10 machines, each with 72~ CPUs. There's a blog post about someone recreating the paper here[1].
Sure, you can't do it on ONE CPU, but the point is to have a cluster of CPUs. It's not the case that you're forced to use nVidia's proprietary stuff in order to do deep learning.
[0]: https://arxiv.org/abs/1712.06567 https://arxiv.org/abs/1712.06567
[1]: https://towardsdatascience.com/paper-repro-deep-neuroevolution-756871e00a66 https://towardsdatascience.com/paper-repro-deep-neuroevoluti...
- blt 8y agoIt's definitely feasible and widely used for reinforcement learning on low dimensional systems, where the neural networks are small and the simulator is more expensive than backprop. On other hand, deep Q learning from pixels on Atari is practically impossible without GPUs.
- tree_of_item 8y agoHuh? I just showed you reinforcement learning from pixels on Atari with CPU...
- gnufx 8y agoFor what it's worth, Cori http://www.nersc.gov/users/data-analytics/data-analytics-2/deep-learning/ http://www.nersc.gov/users/data-analytics/data-analytics-2/d... claimed 15 PF https://arxiv.org/pdf/1708.05256.pdf https://arxiv.org/pdf/1708.05256.pdf