3 ms·
GPUs and CPUs have very different design methodologies. GPUs have very high memory latency--so high in fact, that the basic design of a GPU is to schedule enoug
by jcranmer 9y ago
GPUs and CPUs have very different design methodologies. GPUs have very high memory latency--so high in fact, that the basic design of a GPU is to schedule enough threads per core so that you can hide the latency of memory requests. On top of this, GPUs impose a requirement that you have to have the control flow be more or less convergent between different threads. You also have a high cost to launch a kernel. Ultimately, GPUs require you to have lots of data in a high data-level parallelism to get any efficiency out of them. Fail to meet any of these requirements, and you can't run on a GPU.
A CPU's fundamental design is to try to minimize the latency of any individual instruction, and to try to scavenge any instruction-level parallelism possible. If what you have is a code that is fundamentally serial, with no opportunity for parallelism, a CPU can turn everything else off and ramp up its core frequency to get it done as fast as possible. An example of an unparallelizable code is a binary search: it's a single loop with no instructions outside the critical path, and the inner branches and memory accesses are unpredictable.
Another point to make about parallelism: communication bottleneck is the real killer. Your raw FLOPs performance scales more or less linearly with the number of nodes, but topologies mean your communication bandwidth between nodes scales closer to the sqrt (2D mesh) or n^2/3 (3D mesh). It's easy to make a chip with an obscenely high peak FLOP count, but when you try to program it, you find you can't fill the ALUs fast enough, so most of those are idle.