> But from a programming point of view, CUDA and OpenCL PROVE that you can use SIMD-cores as if they were a bunch of threads. This is nonsense - to beat a CPU

> But from a programming point of view, CUDA and OpenCL PROVE that you can use SIMD-cores as if they were a bunch of threads. This is nonsense - to beat a CPU you have understand that you can't have every thread doing something different. > How to deal with divergence? Just don't use if-statements! Individual threads don't have to avoid if statements. Trying to group threads and SIMD lanes together, only to make all sorts of concessions as to how they are different is silly an counter productive. > We're talking $325 million into a NVidia V100-based supercomputer. It seems like Oak Ridge National Laboratory believes in this methodology. Just because they are using recent GPUs doesn't mean they 'treat SIMD lanes as threads'. It just is not correct to say 'SIMD lanes are just like threads except you can't diverge, you have to run the same instruction, it's better if you don't load from different memory addresses and any non-trivial branching kills performance, including looping a different number of times on different cores'