3 ms·
This is unbelievably dumb. Folks at NVIDIA HQ are no doubt breaking out their best champagne over this news: their Tensor Cores got a few more years of life tha
by m0zg 7y ago
This is unbelievably dumb. Folks at NVIDIA HQ are no doubt breaking out their best champagne over this news: their Tensor Cores got a few more years of life thanks to this decision, and they retain the pricing power as the only provider of practical deep learning acceleration that actually works and has good tooling. I was looking forward to their TPU-like "N" offering in particular. Roughly the throughput of Google TPUv3, better silicon expertise, _and_ I'd be able to buy one. As a an applied scientist, I'd pay good money for 100TFLOPS of bfloat16 throughput, better yet if I can stick 4 of those things into each machine and train things over lunch that'd normally take a week to train. And it was pretty simple: just a systolic array with some extras to efficiently do convolutions, nothing super complicated, which means that unlike the more DSP-like approaches like Habana you don't have to do spend years writing and debugging a specialized compiler. You get to have this throughput _now_. Dumb, dumb move on Intel's part.
- tachyonbeam 7y agoAMD support for PyTorch is getting pretty good from what I understand. They're going to become serious competition soon.
- m0zg 7y agoJust in time for NVIDIA to blow them out of the water completely on both the included RAM (which they're increasing) and performance (+40% if rumors are to be believed) with Ampere. AMD has a looong way to go.
- tachyonbeam 7y agoAMD might still be able to offer competitive price/performance. Even if Nvidia is faster, I'm glad there's competition. For a while, Intel was really dominant, and they really stagnated.
- shaklee3 7y agoJust out of curiosity, why is a V100 with 100TFLOPS of fp16 not good enough?
- m0zg 7y agoBecause: 1. It's not 100TFLOPs - you need fp32 to accumulate dot product, at which point you're getting much less. But even if you do 16/16, there's no way you'll really get near the roof of that roofline model. 2. Each V100 is $7K 3. 4x V100s not only cost as much as a decent car, but require a specialized chassis and a specialized PSU: they're 300W _sustained_ each (substantially more in momentary power consumption), and require a powerful external fan to cool them properly. I want 400TFLOPs of bfloat16 dot product / convolution bandwidth under my desk, in a reasonably quiet, sub-1KW power envelope.
- shaklee3 7y ago1) not true. I have one and hit 100TFLOPS with large enough batches in fp32 accumulate mode. Other benchmarks agree, so I'm not sure what numbers you're suggesting. 4) maybe get a cluster of Jetsons?
- m0zg 7y agohttps://arxiv.org/pdf/1803.04014.pdf https://arxiv.org/pdf/1803.04014.pdf - no matter what they did, they could not go beyond 83TFLOPs in fp16. And that's just matrix multiply. Any kind of deep learning workload is going to be a lot slower than that.
- shaklee3 7y agoI have results showing otherwise. Maybe an old cuda? Here is anandtech showing almost 100TFLOPS, and they didn't every try hard to tune it: https://www.anandtech.com/show/12170/nvidia-titan-v-preview-titanomachy/4 https://www.anandtech.com/show/12170/nvidia-titan-v-preview-...