3 ms·
This has nothing to do with physics. Even a 8700G offers 4 TFLOPs of bfloat16 performance and the NPU is a tiny part of the chip. The only difference between a
by imtringued 3y ago
This has nothing to do with physics. Even a 8700G offers 4 TFLOPs of bfloat16 performance and the NPU is a tiny part of the chip.
The only difference between a GPU and a CPU when it comes to inference is whether it has an inference accelerator or not and how large the on chip SRAM is and how large the memory bandwidth is.
The accelerators are going to be the same on a GPU or CPU so you're left with differences in SRAM and whether the CPU is connected to DDR, GDDR or HBM memory.
That again has very little to do with physics. There is no physical law that prevents you from hooking up GDDR to a CPU if it has a GDDR memory controller.
You are also ignoring the very real possibility of LPDDR PIM which would negate most of the benefits of GPUs since the most expensive operations never leave the memory chip in the first place. HBM PIM only made a GPU 2x times faster vs regular HBM, but it makes CPUs 10x times faster using LPDDR PIM.