3 ms·
Right, but the tensor cores should be about 10x faster on the compute side, and about 2x the memory bandwidth. GEMM is usually constrained by compute, which is
by shaklee3 8y ago
Right, but the tensor cores should be about 10x faster on the compute side, and about 2x the memory bandwidth. GEMM is usually constrained by compute, which is why the tensor cores exist.
These benchmarks are for training, so the expectation is that they are running them in fp16 all the way through. Also, tensor cores can accumulate in fp32 registers with a slight hit to performance.
- KaoruAoiShiho 8y agoTuring tensor cores are not as fast as Volta ones. You won't see the 10x number repeated on any turing marketing material.
- shaklee3 8y agoWhat's your source for that? They should be the same speed, but more capable. You absolutely see the 8x in their marketing: https://devblogs.nvidia.com/nvidia-turing-architecture-in-depth/ https://devblogs.nvidia.com/nvidia-turing-architecture-in-de...
- sabalaba 8y agoIs there a benchmark you've seen that matches the claimed 10x increase in performance for the Tensor Cores? NVIDIA hype train can sometimes make it difficult to find hard numbers.
- shaklee3 8y agoI've done my own benchmarks where I've hit over 100 TFLOPS on the V100, and that's about 85% of the peak theoretical throughout of them. Granted the matrix size needs to be large enough, but it's definitely doable. Anandtech also showed similar results in their V100 review. I haven't yet seen a comparable SGEMM done on the 2080Ti, so I don't know how it'll compare. I have some coming in at the end of the month though, so I should know soon.
- dimitry12 8y agoTitan V in my testing only approached 100+TFlops in purely synthetic large matmuls. With carefully tuned Transformer (matmul-heavy!) I could only make twice as fast as 1080ti (at 4 times the price). The only undisputable benefit was using double batch size.