4 ms·
From first looks, there is little doubt that that NVIDIA's Volta architecture is a monster and will revolutionize the AI and HPC market. But the article seems t
by maldeh 9y ago
From first looks, there is little doubt that that NVIDIA's Volta architecture is a monster and will revolutionize the AI and HPC market. But the article seems to avoid quantifying how 16-point FP operations are beneficial against 32- or 64-bit FP operations in real-world usecases, or how the Caffe2 / NVIDIA architecture provides any significant boost to FP16 in particular, especially apropos to images (or why FP16 is better for image data in general).
I'm interested more in understanding why Caffe2 would outperform Theano, Tensorflow, MXNet, etc. once Volta chipsets are generally available, beyond early pre-release optimization, particularly when most of the front-runners are already leveraging / taking into account NCCL, CuDNN, NVLink, etc. When the burden of adding support for new NVIDIA primitives is so low, what gives Caffe2 an advantage beyond an ephemeral "we were partners with NVIDIA first" one-up that would last for a couple of months at most?
(Apologies in advance if this post sounds overly negative, but I am constantly evaluating the current crop of frameworks for the trade-offs they enforce on the problem space, and a definitive answer would be very helpful.)
- deepnotderp 9y agoCaffe2 seems to be engineered for efficiency as opposed to flexibility. For example, Caffe2 is apparently Facebook's go-to choice for mobile edge device deployment for applications such as style transfer.
- throwaway87213 9y agoEfficient how ? In terms of memory management ?
- eleitl 9y agoTPUs are smaller in terms of Si real estate and burn less power. Also, they can be made much faster.
- sja 9y agoI mean, it uses a static graph definition instead of dynamic graphs (allows for deployment optimizations), but is there anything Caffe2-specific that would give it a leg up over Theano/TensorFlow/whatever other static graph library?
- raverbashing 9y ago16fp is faster, and the increased precision is not needed in most cases (In fact there are even some experiments with a very low number of bits - can't find the link now though)
- buildbot 9y ago16 bit FP operations seem to be more than enough for most networks - in fact for inference, google claims in their TPU paper that most network parameters can be quantized down to 8 integer bits! There is also some work on pure binary networks, xnornet and binarynet. Low precision doesn't seem to effect these networks as long as they are trained with binary weights in the first place.
- user5994461 9y ago>>> I'm interested more in understanding why Caffe2 would outperform [...] A vendor-made benchmark where the vendor is outperforming other vendors. Let's not jump to conclusion on what's really fastest.
- jackmott 9y ago16 bits lets them build something that goes much faster and/or faster per watt and neural nets don't need even 16bits usually. pretty simple!
- endorphone 9y agoOn the topic of FP- https://dennisforbes.ca/index.php/2017/04/11/floating-point-numbers-an-infinite-number-of-mathematicians-enter-a-bar/ https://dennisforbes.ca/index.php/2017/04/11/floating-point-... Most desktop chips offer little advantage for FP16 (at best that the source memory footprint is smaller, maybe better cache hit rates), but newer GPUs can actually blaze. If it has the magnitude/precision needs for a task, it's a big win.
- tachyonbeam 9y agoFP16 multipliers are smaller (in terms of silicon footprint), which means that you can fit more of them on a given silicon die, and they use less power than FP32. The other advantage is that FP16 uses half as much memory, meaning that FP16 operations require half as much memory traffic, which can lead to faster computations. It can also allow you to fit models on your GPU which have twice as many parameters. The reason you don't typically have FP16 on CPUs is that in the CPU world, cache dominates die area and power usage. The floating point ALU component of a CPU is relatively tiny. Still, even in the CPU world, Intel recently added instructions to convert to/from FP16 before operating on it, because they know there is a memory bandwidth advantage in using a smaller floating-point format.