4 ms·

SIMD lanes can absolutely not be treated as threads. Not only can they not diverge in the instructions they execute (obviously) they can't efficiently load fro

SIMD lanes can absolutely not be treated as threads. Not only can they not diverge in the instructions they execute (obviously) they can't efficiently load from different addresses (gather) or write to different addresses (scatter) on AVX. Thinking of SIMD lanes as threads is not likely to be a path that pays off.

dragontamer 8y ago
Performance wise, you are correct. But from a programming point of view, CUDA and OpenCL PROVE that you can use SIMD-cores as if they were a bunch of threads. Yes, there is a performance penalty (an exponentially increasing penalty) on every thread divergence. However, a HUGE set of problems (Deep learning, matrix multiplication, graphics, ray-tracing, encoding, databases: Join-Merge, Join-Hash, etc. etc. ) which have successfully been implemented as "SIMD as if it were threads". How to deal with divergence? Just don't use if-statements! If you can figure out how to cheat if-statements out of your code (or to group "threads" so that they take if-statements all together at the same time), you get HUGE performance on these architectures. And GPU programmers have consistently found more and more ways to "reconverge" threads so that they can be "ganged up" as SIMD-threads in the most efficient manner possible. No, it doesn't work for Chess or other "innately divergent" problems. It doesn't work for many problems. But this "gang up SIMD into threads" works very well for many, many problems. And its easy to do. Case in point: Ray-tracing. Every ray starts off as a camera-ray, but then when it hits an object, it may turn into a diffuse, specular, or sub-surface scattering ray (or many other types, depending on what object you hit). Seems like we have to use an if-statement to continue... But that's wrong! Raytracing becomes GPU-friendly by grouping all specular-rays together, and computing SIMD-gangs on all specular-rays. Then, group all diffuse-rays together, and gang up the threads across those. Etc. etc. Specular rays may bounce off and turn into Diffuse, specular, Sub-surface, or other kinds of rays on each bounce. But just "re-configure" the SIMD-gang on each bounce, and you get huge SIMD-based-benefits. Again: get rid of your if-statements, and you scale. I admit it doesn't work all the time, but it does work a LOT of the time. > Thinking of SIMD lanes as threads is not likely to be a path that pays off. Every supercomputer that is buying a V100 disagrees with you (ie: Summit). There are plenty of useful problems that work under the SIMD-as-threads model. https://en.wikipedia.org/wiki/Summit_(supercomputer) https://en.wikipedia.org/wiki/Summit_(supercomputer) We're talking $325 million into a NVidia V100-based supercomputer. It seems like Oak Ridge National Laboratory believes in this methodology. > they can't efficiently load from different addresses (gather) or write to different addresses (scatter) on AVX. AVX2 has Gather instructions. While Scatter instructions were added to AVX512. See AVX2 instruction: VPGATHERDD
- CyberDildonics 8y ago
  > But from a programming point of view, CUDA and OpenCL PROVE that you can use SIMD-cores as if they were a bunch of threads. This is nonsense - to beat a CPU you have understand that you can't have every thread doing something different. > How to deal with divergence? Just don't use if-statements! Individual threads don't have to avoid if statements. Trying to group threads and SIMD lanes together, only to make all sorts of concessions as to how they are different is silly an counter productive. > We're talking $325 million into a NVidia V100-based supercomputer. It seems like Oak Ridge National Laboratory believes in this methodology. Just because they are using recent GPUs doesn't mean they 'treat SIMD lanes as threads'. It just is not correct to say 'SIMD lanes are just like threads except you can't diverge, you have to run the same instruction, it's better if you don't load from different memory addresses and any non-trivial branching kills performance, including looping a different number of times on different cores'

dragontamer 8y ago

Performance wise, you are correct. But from a programming point of view, CUDA and OpenCL PROVE that you can use SIMD-cores as if they were a bunch of threads. Yes, there is a performance penalty (an exponentially increasing penalty) on every thread divergence. However, a HUGE set of problems (Deep learning, matrix multiplication, graphics, ray-tracing, encoding, databases: Join-Merge, Join-Hash, etc. etc. ) which have successfully been implemented as "SIMD as if it were threads". How to deal with divergence? Just don't use if-statements! If you can figure out how to cheat if-statements out of your code (or to group "threads" so that they take if-statements all together at the same time), you get HUGE performance on these architectures. And GPU programmers have consistently found more and more ways to "reconverge" threads so that they can be "ganged up" as SIMD-threads in the most efficient manner possible. No, it doesn't work for Chess or other "innately divergent" problems. It doesn't work for many problems. But this "gang up SIMD into threads" works very well for many, many problems. And its easy to do. Case in point: Ray-tracing. Every ray starts off as a camera-ray, but then when it hits an object, it may turn into a diffuse, specular, or sub-surface scattering ray (or many other types, depending on what object you hit). Seems like we have to use an if-statement to continue... But that's wrong! Raytracing becomes GPU-friendly by grouping all specular-rays together, and computing SIMD-gangs on all specular-rays. Then, group all diffuse-rays together, and gang up the threads across those. Etc. etc. Specular rays may bounce off and turn into Diffuse, specular, Sub-surface, or other kinds of rays on each bounce. But just "re-configure" the SIMD-gang on each bounce, and you get huge SIMD-based-benefits. Again: get rid of your if-statements, and you scale. I admit it doesn't work all the time, but it does work a LOT of the time. > Thinking of SIMD lanes as threads is not likely to be a path that pays off. Every supercomputer that is buying a V100 disagrees with you (ie: Summit). There are plenty of useful problems that work under the SIMD-as-threads model. https://en.wikipedia.org/wiki/Summit_(supercomputer) https://en.wikipedia.org/wiki/Summit_(supercomputer) We're talking $325 million into a NVidia V100-based supercomputer. It seems like Oak Ridge National Laboratory believes in this methodology. > they can't efficiently load from different addresses (gather) or write to different addresses (scatter) on AVX. AVX2 has Gather instructions. While Scatter instructions were added to AVX512. See AVX2 instruction: VPGATHERDD

CyberDildonics 8y ago
> But from a programming point of view, CUDA and OpenCL PROVE that you can use SIMD-cores as if they were a bunch of threads. This is nonsense - to beat a CPU you have understand that you can't have every thread doing something different. > How to deal with divergence? Just don't use if-statements! Individual threads don't have to avoid if statements. Trying to group threads and SIMD lanes together, only to make all sorts of concessions as to how they are different is silly an counter productive. > We're talking $325 million into a NVidia V100-based supercomputer. It seems like Oak Ridge National Laboratory believes in this methodology. Just because they are using recent GPUs doesn't mean they 'treat SIMD lanes as threads'. It just is not correct to say 'SIMD lanes are just like threads except you can't diverge, you have to run the same instruction, it's better if you don't load from different memory addresses and any non-trivial branching kills performance, including looping a different number of times on different cores'