4 ms·
The fp16 results versus the 1080ti are somewhat surprising. The author specifically pointed out that they are using tensor cores were possible. I would have exp
by shaklee3 8y ago
The fp16 results versus the 1080ti are somewhat surprising. The author specifically pointed out that they are using tensor cores were possible. I would have expected fp16 would be more than 100% faster than the 1080ti if they were using tensor cores. Can anyone explain that?
- pavanky 8y agoNot everything in the compute pipeline is going to be converted to fp16 operations. Anytime you are doing accumulation or exponentials, you would have to have it in fp32. There was a good talk from NVIDIA at last years GTC: http://on-demand.gputechconf.com/gtc/2018/presentation/s8923-training-neural-networks-with-mixed-precision-theory-and-practice.pdf http://on-demand.gputechconf.com/gtc/2018/presentation/s8923... Here is another relevant blog post: https://devblogs.nvidia.com/mixed-precision-training-deep-neural-networks/ https://devblogs.nvidia.com/mixed-precision-training-deep-ne... EDIT: Also not everything in the training loop is a matrix multiplication where tensor cores are useful.
- shaklee3 8y agoRight, but the tensor cores should be about 10x faster on the compute side, and about 2x the memory bandwidth. GEMM is usually constrained by compute, which is why the tensor cores exist. These benchmarks are for training, so the expectation is that they are running them in fp16 all the way through. Also, tensor cores can accumulate in fp32 registers with a slight hit to performance.
- KaoruAoiShiho 8y agoTuring tensor cores are not as fast as Volta ones. You won't see the 10x number repeated on any turing marketing material.
- shaklee3 8y agoWhat's your source for that? They should be the same speed, but more capable. You absolutely see the 8x in their marketing: https://devblogs.nvidia.com/nvidia-turing-architecture-in-depth/ https://devblogs.nvidia.com/nvidia-turing-architecture-in-de...
- sabalaba 8y agoIs there a benchmark you've seen that matches the claimed 10x increase in performance for the Tensor Cores? NVIDIA hype train can sometimes make it difficult to find hard numbers.
- shaklee3 8y agoI've done my own benchmarks where I've hit over 100 TFLOPS on the V100, and that's about 85% of the peak theoretical throughout of them. Granted the matrix size needs to be large enough, but it's definitely doable. Anandtech also showed similar results in their V100 review. I haven't yet seen a comparable SGEMM done on the 2080Ti, so I don't know how it'll compare. I have some coming in at the end of the month though, so I should know soon.
- dimitry12 8y agoTitan V in my testing only approached 100+TFlops in purely synthetic large matmuls. With carefully tuned Transformer (matmul-heavy!) I could only make twice as fast as 1080ti (at 4 times the price). The only undisputable benefit was using double batch size.
- ece 8y agoI think a good answer to this is here talking about the tensor cores on the Nvidia Titan V: https://github.com/tensorflow/tensorflow/issues/15897#issuecomment-356503342 https://github.com/tensorflow/tensorflow/issues/15897#issuec... If you can write a better optimized network, go ahead. But like in SSE2 vs AVX2 vs AVX512 benchmarks, FP performance on paper doesn't always translate into better real world FP performance. Now if Nvidia had switched to HBM2 like Google TPU2, it might be different.
- howscrewedami 8y agoIt's still about 60% faster than the 1080 ti, which is huge in my opinion.