2 ms·

I found it very weird that GPU shader languages (and CUDA) have solved the vectorization problem so successfully, it's almost unheard of for people to write str

I found it very weird that GPU shader languages (and CUDA) have solved the vectorization problem so successfully, it's almost unheard of for people to write straight SIMD vector code (not even sure if you can). Yet on the CPU, a far more flexible architecture, it's still almost completely unsolved. Autovectorization is unreliable, more like a 'take what you can get' or 'toy example' nice boost than something to be relied on. People even say that intrinsics tend to suck bad compared to hand-written assembly, which is the only true way. Almost nobody uses tools like ispc, which I find puzzling.

mhh__ 3y ago
Shaders are for different hardware than regular code, both in the amount of independent work happening all the time but also the memory hierarchy is designed for very different workloads. I think it's the memory than explains why one has been more successful. A more practical reason in terms of ergonomics is probably that by the time you have a problem that fits a different programming model but still compiles to SIMD you're basically in GPU territory already.
JonChesterfield 3y ago
There's a degree of convergent evolution between CPUs and GPUs. The basic question is "what shall we do while waiting for memory". Branch prediction and speculative execution gives you a CPU, swap to another thread gives you a GPU. The wide vector unit vs scalar unit choice is sort of orthogonal to that. I'd guess you reach for a GPU if the problem decomposes into lots of similar independent operations, at which point vector units are also what you want. And a CPU when there are deep data dependencies, where you're less likely to be able to pack the vector. Implicitly vectorised code on a CPU might be interesting given the increasing variety of avx instructions. I'm interested in scalar workloads on a GPU but there's a credible chance performance won't work out well enough.

mhh__ 3y ago

Shaders are for different hardware than regular code, both in the amount of independent work happening all the time but also the memory hierarchy is designed for very different workloads. I think it's the memory than explains why one has been more successful. A more practical reason in terms of ergonomics is probably that by the time you have a problem that fits a different programming model but still compiles to SIMD you're basically in GPU territory already.

JonChesterfield 3y ago

There's a degree of convergent evolution between CPUs and GPUs. The basic question is "what shall we do while waiting for memory". Branch prediction and speculative execution gives you a CPU, swap to another thread gives you a GPU. The wide vector unit vs scalar unit choice is sort of orthogonal to that. I'd guess you reach for a GPU if the problem decomposes into lots of similar independent operations, at which point vector units are also what you want. And a CPU when there are deep data dependencies, where you're less likely to be able to pack the vector. Implicitly vectorised code on a CPU might be interesting given the increasing variety of avx instructions. I'm interested in scalar workloads on a GPU but there's a credible chance performance won't work out well enough.