6 ms·

It's interesting that SIMD came to the mainstream 25 years ago and compilers, our apps and PL tech are still quite far from effectively utilizing it outside non

It's interesting that SIMD came to the mainstream 25 years ago and compilers, our apps and PL tech are still quite far from effectively utilizing it outside nonportable manually coded SIMD aware compute kernels in glorified assembler. There are some exceptions like ispc and GPU languages like OpenCL and Futhark (GPU people say "cores" when they mean SIMD lanes!)...

zozbot234 5y ago
LLVM does a pretty good job of vectorizing code these days, and pretty much everything is using it. And making vectorization more effective is one of the most workable avenues for improving compiler tech.
- fulafel 5y ago
  Given the constraints llvm frequently does a relatively ok job with the pieces it has to work with, but the results are usually poor in absolute terms vs what manual optimizations would allow. The language semantics of c/c++ don't support the required leeway of optimization transformations (eg optimizing data layout hand in hand with it's access patterns, parallel semantics).
  - mandarax8 5y ago
    Are there any languages you know that do support these kinds of transformations?
    - fulafel 5y ago
      Besides the ones mentioned in the other comment, Halide I think is a good model of the kind of approach needed, but that is somewhat domain-specific. So expressing the problem in a sufficiently abstract form that the compiler has many freedoms in organizing how the computation in concrete terms takes place. The IPSC approach seems solid as well even though it's much less ambitious and doesn't do data layout optimizations. An important nontechnical problem with these is that there is such a long way to travel for a language to reach widespread adoption, needing credible long term sponsorship and commitment from a broad shouldered user community or private organisation, etc.
  - inetknght 5y ago
    > c++ don't support the required leeway of optimization transformations What transformations do you think C++ doesn't support? I've got a _lot_ of experience in C++ and I've found that it's almost always the case that whatever library being used needs to be rewritten for a small change. Rewriting libraries is rarely approved. And further, there's often very little unit testing being done towards that end too.
    - fulafel 5y ago
      I meant automatic optimizations done by the compiler, and limitations imposed by the language semantics.
fleabitdev 5y ago
I recently experimented with Rust's std::simd library prototype, and it gave me the same sense of wasted potential. My program performed complicated processing of large 8-bit grayscale images. Naive use of portable SIMD made several important parts of the algorithm run ten times faster than my integrated-GPU implementation! (OpenGL ES 3.0 over ANGLE on an Intel HD Graphics 520) I think the sheer power of modern SIMD has slipped past most peoples' radar. A multicore processor with AVX2 has number-crunching throughput comparable to a low-end GPU, and for some workloads, the extra flexibility of the CPU can unlock an additional order-of-magnitude performance uplift. (The comparison may be a little different if you're programming the GPU with modern compute shaders - I'm not sure.) Portable SIMD programming is also just fun! Trying to keep my working set within the 32 KiB L1 cache, trying to keep my data packed within individual 8-bit lanes, and lacking access to some basic operations like integer multiplication and division - it almost feels like programming for a retro games console.
- Royi 5y ago
  Have you looked at [ISPC - Intel SPMD Program Compiler][0]? [0]: https://github.com/ispc/ispc
- shakow 5y ago
  I concur. I recently ported some bioinfo computations from scalar to SIMD with primitives from core::arch, and the gain was an immediate ~10×. It is a shame that one needs to fiddle with RUSTC_FLAGS just for SIMD extensions to trigger though :/
- dahart 5y ago
  > it almost feels like programming for a retro games console. I get this feeling writing CUDA code all the time, having to keep my stack frames and local data small, figuring out how to even out and time-slice the work. It’s a little like Arduino coding in a way, it’s just that instead of one of them running at 16Mhz, I have ten thousand of them running at 2Ghz (and as a bonus they have floating point superpowers).
torginus 5y ago
The really strange thing about this is that GPU shaders are essentially SIMD programs - but nobody, except for some real hardcore gamedevs, program SIMD assembly directly, the compiler handles it for you. There's ISPC, which is basically a C-like language and SIMD compiler, that translates it's scalar code into SIMD assembly, but other than that, SIMD code tends to be hand rolled.
antpls 5y ago
SIMD instructions as we know them today do not make a consensus among the actors and architectures. See : https://www.phoronix.com/scan.php?page=news_item&px=Linus-Torvalds-On-AVX-512 https://www.phoronix.com/scan.php?page=news_item&px=Linus-To... That makes them harder to implement in compilers and in turns to democratize SIMD. There is an article on the web explaining the purpose of SVE/SVE2 ("Scalable Vector Extension"), which is supposed to be the successor of SIMD on ARM : https://levelup.gitconnected.com/armv9-what-is-the-big-deal-4528f20f78f3?gi=5fcfb9df90cf https://levelup.gitconnected.com/armv9-what-is-the-big-deal-... Extract : "[...] the addition of SIMD instructions has led to an explosion in the number of instructions, especially for x86. And of course not every x86 processor will support all these instructions. Only the newer ones will support AVX-512. The beauty of SVE is that the same code will work for both the super-computer and the cheap phone. That is not possible with the x86 SIMD instructions." There is also a Java proposal to use SVE as the way of doing SIMD in the Java world : https://openjdk.java.net/jeps/417 https://openjdk.java.net/jeps/417 The same principle will be extended on ARM to matrices with "Scalable Matrix Extension" : https://community.arm.com/arm-community-blogs/b/architectures-and-processors-blog/posts/scalable-matrix-extension-armv9-a-architecture https://community.arm.com/arm-community-blogs/b/architecture... We can speculate that everyone will migrate to ARM / RISC-V at some point, or x86 will have similar instructions.
- janwas 5y ago
  It's indeed nice that SVE2 binaries can run on differing HW, and fragmentation will be less because it's required in ARMv9. > the same code will work for both the super-computer and the cheap phone. That is not possible with the x86 SIMD instructions Actually, when the code is expressed using "portable intrinsics" (https://github.com/google/highway https://github.com/google/highway), the source code looks the same but compiles to SSE4/AVX2/AVX-512 and NEON,SVE,SVE2 and RISC-V V instructions. Disclosure: I am the main author of this library.

zozbot234 5y ago

LLVM does a pretty good job of vectorizing code these days, and pretty much everything is using it. And making vectorization more effective is one of the most workable avenues for improving compiler tech.

fulafel 5y ago
Given the constraints llvm frequently does a relatively ok job with the pieces it has to work with, but the results are usually poor in absolute terms vs what manual optimizations would allow. The language semantics of c/c++ don't support the required leeway of optimization transformations (eg optimizing data layout hand in hand with it's access patterns, parallel semantics).
- mandarax8 5y ago
  Are there any languages you know that do support these kinds of transformations?
  - fulafel 5y ago
    Besides the ones mentioned in the other comment, Halide I think is a good model of the kind of approach needed, but that is somewhat domain-specific. So expressing the problem in a sufficiently abstract form that the compiler has many freedoms in organizing how the computation in concrete terms takes place. The IPSC approach seems solid as well even though it's much less ambitious and doesn't do data layout optimizations. An important nontechnical problem with these is that there is such a long way to travel for a language to reach widespread adoption, needing credible long term sponsorship and commitment from a broad shouldered user community or private organisation, etc.
- inetknght 5y ago
  > c++ don't support the required leeway of optimization transformations What transformations do you think C++ doesn't support? I've got a _lot_ of experience in C++ and I've found that it's almost always the case that whatever library being used needs to be rewritten for a small change. Rewriting libraries is rarely approved. And further, there's often very little unit testing being done towards that end too.
  - fulafel 5y ago
    I meant automatic optimizations done by the compiler, and limitations imposed by the language semantics.

fleabitdev 5y ago

I recently experimented with Rust's std::simd library prototype, and it gave me the same sense of wasted potential. My program performed complicated processing of large 8-bit grayscale images. Naive use of portable SIMD made several important parts of the algorithm run ten times faster than my integrated-GPU implementation! (OpenGL ES 3.0 over ANGLE on an Intel HD Graphics 520) I think the sheer power of modern SIMD has slipped past most peoples' radar. A multicore processor with AVX2 has number-crunching throughput comparable to a low-end GPU, and for some workloads, the extra flexibility of the CPU can unlock an additional order-of-magnitude performance uplift. (The comparison may be a little different if you're programming the GPU with modern compute shaders - I'm not sure.) Portable SIMD programming is also just fun! Trying to keep my working set within the 32 KiB L1 cache, trying to keep my data packed within individual 8-bit lanes, and lacking access to some basic operations like integer multiplication and division - it almost feels like programming for a retro games console.

Royi 5y ago
Have you looked at [ISPC - Intel SPMD Program Compiler][0]? [0]: https://github.com/ispc/ispc
shakow 5y ago
I concur. I recently ported some bioinfo computations from scalar to SIMD with primitives from core::arch, and the gain was an immediate ~10×. It is a shame that one needs to fiddle with RUSTC_FLAGS just for SIMD extensions to trigger though :/
dahart 5y ago
> it almost feels like programming for a retro games console. I get this feeling writing CUDA code all the time, having to keep my stack frames and local data small, figuring out how to even out and time-slice the work. It’s a little like Arduino coding in a way, it’s just that instead of one of them running at 16Mhz, I have ten thousand of them running at 2Ghz (and as a bonus they have floating point superpowers).

torginus 5y ago

The really strange thing about this is that GPU shaders are essentially SIMD programs - but nobody, except for some real hardcore gamedevs, program SIMD assembly directly, the compiler handles it for you. There's ISPC, which is basically a C-like language and SIMD compiler, that translates it's scalar code into SIMD assembly, but other than that, SIMD code tends to be hand rolled.

antpls 5y ago

SIMD instructions as we know them today do not make a consensus among the actors and architectures. See : https://www.phoronix.com/scan.php?page=news_item&px=Linus-Torvalds-On-AVX-512 https://www.phoronix.com/scan.php?page=news_item&px=Linus-To... That makes them harder to implement in compilers and in turns to democratize SIMD. There is an article on the web explaining the purpose of SVE/SVE2 ("Scalable Vector Extension"), which is supposed to be the successor of SIMD on ARM : https://levelup.gitconnected.com/armv9-what-is-the-big-deal-4528f20f78f3?gi=5fcfb9df90cf https://levelup.gitconnected.com/armv9-what-is-the-big-deal-... Extract : "[...] the addition of SIMD instructions has led to an explosion in the number of instructions, especially for x86. And of course not every x86 processor will support all these instructions. Only the newer ones will support AVX-512. The beauty of SVE is that the same code will work for both the super-computer and the cheap phone. That is not possible with the x86 SIMD instructions." There is also a Java proposal to use SVE as the way of doing SIMD in the Java world : https://openjdk.java.net/jeps/417 https://openjdk.java.net/jeps/417 The same principle will be extended on ARM to matrices with "Scalable Matrix Extension" : https://community.arm.com/arm-community-blogs/b/architectures-and-processors-blog/posts/scalable-matrix-extension-armv9-a-architecture https://community.arm.com/arm-community-blogs/b/architecture... We can speculate that everyone will migrate to ARM / RISC-V at some point, or x86 will have similar instructions.

janwas 5y ago
It's indeed nice that SVE2 binaries can run on differing HW, and fragmentation will be less because it's required in ARMv9. > the same code will work for both the super-computer and the cheap phone. That is not possible with the x86 SIMD instructions Actually, when the code is expressed using "portable intrinsics" (https://github.com/google/highway https://github.com/google/highway), the source code looks the same but compiles to SSE4/AVX2/AVX-512 and NEON,SVE,SVE2 and RISC-V V instructions. Disclosure: I am the main author of this library.