3 ms·
The CPUs are good at serial code and GPUs are good at parallel code is kind of true but something of an approximation. Assume equivalent power budget in the rou
by JonChesterfield 3y ago
The CPUs are good at serial code and GPUs are good at parallel code is kind of true but something of an approximation. Assume equivalent power budget in the roughly hundreds of watts range, then:
A CPU has ~100 "cores" each running one (and-a-hyperthread). independent things, and it hides memory latency by branch prediction and pipelining.
A GPU has ~100 "compute units", each running ~80 independent things interleaved, and it hides memory latency by executing the next instruction from one of the other 80 things.
Terminology is a bit of a mess, and the CPU probably has a 256bit wide vector unit while the GPU probably has a 2048bit wide vector unit, but from a short distance the two architectures look rather similar.
- bee_rider 3y agoI’m always surprised there isn’t a movement toward pairing a few low latency cores with a large number of high throughput cores. Surround single Intel P core with a bunch of E cores. Then, hanging off the E cores, stick a bunch of iGPU cores and/or AVX-512 units. Call it Xeon Chi.
- softfalcon 3y agoNeat idea, probably even viable! I think they may have a hurdle of getting folks to buy into the concept though. I imagine it would be analogous to how Arria FPGA’s were included with certain Xeon CPU’s. Which further backs up your point that this could happen in the near future!
- Const-me 3y agoI think one possible reason for that, ideally these things need different memory. If you use high-bandwidth high-latency GDDR memory, CPU cores will underperform due to high latency, like there: https://www.tomshardware.com/reviews/amd-4700s-desktop-kit-review-ps5-cpu https://www.tomshardware.com/reviews/amd-4700s-desktop-kit-r... If you use low-latency memory, GPU cores will underperform due to low bandwidth, see modern AMD APUs with many RDNA3 cores connected to DDR5 memory. On paper, Radeon 780M delivers up to 9 FP32 TFLOPS, the figure is close to desktop version of Radeon RX 6700 which is substantially faster in gaming.
- bee_rider 3y agoHmm, that is a good point. Since it is a dream-computer anyway, maybe we can do 2.5d packaging; put the ddr memory right on top so the P cores can reach it quickly, then surround the whole thing with GDDR.
- pixelpoet 3y agoYou mean like an iGPU? Edit: Oh, thanks for the downvote, with no discussion of the question. I'll just sit here quietly with my commercial OpenCL software that happily exploits these vector units attached to the normal CPU cores.
- bee_rider 3y agoI’m not sure who downvoted; I think it isn’t possible to downvote a response to one’s comment. I did decide not to engage because “you mean like <very common well known thing>” seemed a bit brusque and dismissive, commenting on this site is just for fun, so I don’t really see the point in continuing a conversation that seems like it is getting off on the wrong foot.
- mmoskal 3y agoGPU has 10x the memory bandwidth of the CPU though, which becomes relevant for the LLMs where you essentially have to read the whole memory (if you're batching optimally, that is using all the memory either for weights or for KV cache) to produce one token of output.