3 ms·
> We know CPU is about 50 times slower than GPU for transformer models Where's this heuristic from? Seems handy if true.
by MiroF 6y ago
> We know CPU is about 50 times slower than GPU for transformer models
Where's this heuristic from? Seems handy if true.
- liuliu 6y agoIt is from my own experience. But it looks like inference is a bit faster on CPU: https://medium.com/huggingface/benchmarking-transformers-pytorch-and-tensorflow-e2917fb891c2 https://medium.com/huggingface/benchmarking-transformers-pyt...
- hansvm 6y agoThose figures make sense as a rough ballpark -- max FLOPS estimates including impacts from FMA and whatnot roughly correspond to the throughput we would expect for a compute-bound linear algebra problem, and current top of the line consumer grade GPUs are 15-30x better in that metric than equivalent CPUs.