4 ms·
> they speed up both training and evaluation by a factor of 100 The factor is more like 5-10x for truly optimized CPU versus GPU in fp32 (e.g., nnpack on CPU v
by jhj 10y ago
> they speed up both training and evaluation by a factor of 100
The factor is more like 5-10x for truly optimized CPU versus GPU in fp32 (e.g., nnpack on CPU versus cuDNN).
- hooloovoo_zoo 10y agoI'm basing that number on https://github.com/jcjohnson/cnn-benchmarks https://github.com/jcjohnson/cnn-benchmarks. I'm sure the exact number depends on the cpu/gpu/algorithm.
- jhj 10y agoim2col + sgemm on CPU as in Caffe for instance is really slow; you are heavily penalized for extra memory traffic and the sgemm tile sizes are probably not well tuned for the problem size at hand. At the roofline of performance, the difference in both mem b/w and arithmetic throughput between CPU and GPU is only 5-10x (for fp32, Pascal fp16 is a different story of course), and proper implementations on the CPU will get you there. https://github.com/Maratyszcza/NNPACK https://github.com/Maratyszcza/NNPACK
- dharma1 10y agonnpack is excellent - but also very new. Noticed it just got NEON optimisations for Arm- do you know if it's possible to use it together with OpenCL on mobile for combined CPU/GPU inference?