3 ms·
The Volta Tensor Cores aren't released yet, and I haven't played with Google's TPUs at all. But NVidia markets Tensor cores as: > New Tensor Cores designed sp
by dragontamer 9y ago
The Volta Tensor Cores aren't released yet, and I haven't played with Google's TPUs at all.
But NVidia markets Tensor cores as:
> New Tensor Cores designed specifically for deep learning deliver up to 12x higher peak TFLOP/ss for training, and 6x higher peak TFLOP/s for inference
https://devblogs.nvidia.com/parallelforall/cuda-9-features-revealed/ https://devblogs.nvidia.com/parallelforall/cuda-9-features-r...
I wouldn't be surprised if they were inflating the numbers slightly, as is common in a lot of marketing material.
- dharma1 9y agoYou can use Tensor Cores on AWS p3 instances (v100) now. Just doesn't seem to be a whole lot of ML framework support for mixed precision training with CUDA 9 yet https://devblogs.nvidia.com/parallelforall/programming-tensor-cores-cuda-9/ https://devblogs.nvidia.com/parallelforall/programming-tenso... Google's TPUs also do low precision training (with some special version of TF). https://cloud.google.com/tpu/ https://cloud.google.com/tpu/