5 ms·
There is zero chance that Intel can add "good enough" functionality to their CPUs without changing their memory architecture. A high end i7 might have 35 GB/s o
by modeless 10y ago
There is zero chance that Intel can add "good enough" functionality to their CPUs without changing their memory architecture. A high end i7 might have 35 GB/s of memory bandwidth. Even the biggest, most ridiculously expensive ($6000) Xeons have only about 100 GB/s. Nvidia's GTX 1080 has 320 GB/s. Their P100 has 732 GB/s.
The threat to Nvidia from Intel is in the Nervana chips they recently acquired. Those are presumably using HBM2 and could potentially beat GPUs for neural net training performance.
- bonzini 10y ago100 GB/s is the bandwidth of six DDR4 channels. Knights Landing has 500 GB/s of local memory bandwidth, plus the same 100 GB/s of a Xeon for DDR4. If future Xeons integrated RAM like Knights Landing does, this would make GPUs less interesting. But KNL can already be used as the main CPU, if I remember correctly.
- geezerjay 10y agoYou're ignoring the main factor: cost. Intel's offering costs as high as €4000 per processor, which come with a meager 8 cores. NVidia's offering sells, right now, for less than €1000 a pop.
- daemin 10y agoWhat about ongoing electricity costs? IIRC my graphics card chews up more power than my CPU.
- geezerjay 10y agoA delta of $3000 per component buys you a whole lot of electricity. Furthermore, the performance of each of NVidia's GPUs falls somewhere between 5 and 20 teraFLOPs. A Haswell Xeon gets you about half a teraFLOP for around 1/3 of the power consumption of a NVidia Tesla P100 GPU.
- bonzini 10y agoI am not sure where you get those numbers. Both a Knights Landing Xeon Phi and a Tesla K80 are about 3 teraflops for $5000. Comparing GPUs with Xeons makes no sense.
- imaginenore 10y agoThat's just moving the data around. CPUs can't do any meaningful (like AI training) data processing at these speeds, as far as I know. GPUs probably can do something simple at these speeds, but I doubt AI training can be done even at 320GB/s on a single GPU.
- LoSboccacc 10y agomoving large data around is the whole issue of deep learning.
- imaginenore 10y agoNo, it's not. You have to apply the data to the evolving network. Can you really do it at 320GB/s?
- railorsi 10y agoGood question. If you have enough cores - maybe? Does anyone have a precise idea?
- hedgehog 10y agoMany inference tasks can fit in cache or eDRAM. The SIMD units are not fasts enough to make it a big problem so long as the code is well-optimized. On the Nervana front, outrunning a GPU for neural nets is not that hard with an ASIC. I checked your profile, I bet your employer knows something about that :)
- boulos 10y agoA modern Xeon has 10s of MBs of cache, which has drastically higher memory bandwidth. What matters is how big the matrices (the "model") is not the input data. With native 8-bit or 16-bit support, you can fit an awful lot of model in 30MB. I'd recommend reading this paper about matrix math on GPUs from friends of mine way back in the day: https://graphics.stanford.edu/papers/gpumatrixmult/gpumatrixmult.pdf https://graphics.stanford.edu/papers/gpumatrixmult/gpumatrix... . While NVIDIA has built much larger register files and L2 caches since then, a modern Xeon still is unbeatable when something fits in L2 or even L3 cache. I believe streaming the input data in to update the weights is usually done with non Cache polluting instructions (mm_stream equivalents with the NT hint), so it's not hard to keep it fed.
- feelix 10y agoIn general neural nets today are so slow (due being computational resource hungry, in clock cycles, memory, and memory bandwidth) that even 4 of the fastest GPU's money can buy stacked together still take days or weeks to train a production quality model. Trying to peg this onto one of their CPU's will have a result much like they have had in pegging the graphics processing stuff onto their CPU's. That is, very meagre results that is fitting for only the lowest resource intensive things that a consumer may want to do. 30MB to hold a model in is almost nothing. Even 12GB is insufficient to train something like imagenet. But in the same vein as their iGPU implementation, it may be just enough to satisfy the requirements of the average consumer, and still be able to make a large dent in Nvidia's market share.
- spangry 10y agoUnless you've got extremely low I/O latency, allowing you to work in smaller batches in parallel without loosing all of your parallelisation speedup to the 'tax' you pay when loading data & instructions from RAM to <insert processing unit here>. Although I have no idea what that implies for the hardware discussed above, as it's way over my head.
- modeless 10y agoSOTA models are much, much larger than 30 MB. You can't use 8 bits for training (yet?), and Intel doesn't support 16-bit float (they need to catch up with Nvidia on that, it's really a travesty, we could have used it years ago for audio and imaging). Besides, the top end Xeons with the most cache are fully 10 times more expensive than a GTX 1080.