4 ms·
It is from my own experience. But it looks like inference is a bit faster on CPU: https://medium.com/huggingface/benchmarking-transformers-pytorch-and-tensorflo
by liuliu 6y ago
It is from my own experience. But it looks like inference is a bit faster on CPU: https://medium.com/huggingface/benchmarking-transformers-pytorch-and-tensorflow-e2917fb891c2 https://medium.com/huggingface/benchmarking-transformers-pyt...
- hansvm 6y agoThose figures make sense as a rough ballpark -- max FLOPS estimates including impacts from FMA and whatnot roughly correspond to the throughput we would expect for a compute-bound linear algebra problem, and current top of the line consumer grade GPUs are 15-30x better in that metric than equivalent CPUs.