You have nailed it; I agree this is the biggest reason why the idea of CPU scaling matching current GPU chips is exciting. Many other replies speak of the diff

You have nailed it; I agree this is the biggest reason why the idea of CPU scaling matching current GPU chips is exciting. Many other replies speak of the difference between “GPU” and “CPU” without describing it—it’s thread divergence. Processing stream-like data (network streams; compilers and other parsers; human input devices) can branch often and unpredictably using normal algorithms. Think for example how you would tokenize JSON without branching. A single GPU core is quite slow for the general case anyway. They are mostly good at math and bitwise operations. They were able to get this way because of the assumption that there would not be thread divergence for the primary workload, i.e. matrix math on contiguous blocks of memory, which harkens back to their origin as actual Graphics Processing Units. If thread divergence weren’t such a killer for branching programs, you could probably write anything in CUDA. It’s not very limited; you can write most anything that can be expressed in non-exotic C. A lot the work has to do with schlepping textures around, which is an artifact of the expectation that your code will run at a distance from the messy, branching, unpredictable main program state, so everything (textures, shaders, etc.) should be loaded, processed, and ready to go when it’s frame-buffer time.