If by processing cores you mean Streaming Multiprocessors (SMs), keep in mind that each SM is 128 threads, so 16 SMs is 2k threads - or 2048 single precision ma

If by processing cores you mean Streaming Multiprocessors (SMs), keep in mind that each SM is 128 threads, so 16 SMs is 2k threads - or 2048 single precision math ops per cycle. The 32 core avx512-enabled CPU is 512 math ops per cycle, if you have one FMA unit per core (or 1k ops/cycle if you have 2 FMA units/core). Note that modern NVIDA GPUs like the 5090 are actually more like 170 SMs on a chip, or 21,760 flop/cycle, or ~20-40x more ops/cycle than your example CPU. Put GDDR6 vs DDR5 memory on top of that, and it’s easy to see why the GPU can churn through math so fast … as long as it’s GPU-y. For stuff the GPU does well, the CPU typically can’t compete, the GPU is often more than 10x faster. But GPU-y tasks are a subset, and there are CPU-y things the GPU can’t compete on, despite (or even because of) the thread count discrepancy.