6 ms·
You got that last sentence right: I’m not aware of any apps which can take advantage of even half this many cores. Intel is deficient in terms of memory bandwid
by 0xbear 9y ago
You got that last sentence right: I’m not aware of any apps which can take advantage of even half this many cores. Intel is deficient in terms of memory bandwidth, so most things begin to flatten out at 8 physical cores or so, and take a nosedive when you attempt to use all of them.
While you are right that there are very few apps able to take advantage of quad GPUs, for anyone who does serious number crunching it’s not a problem at all to use all of them using popular deep learning frameworks. They aren’t just for deep learning, you can do a lot more with them. And e.g. PyTorch will automatically parallelize certain things across multiple GPUs for you, with near linear speed up.
- mugsie 9y agoYeah - my use case is kind of specialised, but I could use them all, asking as I can use NUMA to pin memory to physical CPUs. I wpild be interested to see how pytorch scales GPU wise, relative to CPU clock speed. I am sure there is a point where it will fall off, but not having a 4 GPU system I can't check :)
- 0xbear 9y agoIt’s basically linear as long as CPU can feed the GPU with data. That’s because there’s no cross-GPU communication during the expensive part of the computation (ie forward and backward pass of a neural network). You shove data in, GPU computes gradient, you apply weight updates and redistribute new weights (which are usually not that large). Running the same workload on even the highest end general purpose CPU is glacial compared to even a modestly priced Pascal GPU. That said, there is a limitation: you can’t really have more than 16GB per GPU today, no matter how much you pay, and GPUs can’t do branchy workloads like SQL, compression, etc.