3 ms·

At the level the article look at, it is not even worth doing better, since the potential gain is minuscule compared to the available disk and memory, and this h

At the level the article look at, it is not even worth doing better, since the potential gain is minuscule compared to the available disk and memory, and this has been the case at least since the late 80s. That does not mean that it is always better to let the compiler do all optimizations. There are still cases, especially where clever uses of SIMD instructions can pay off. This is especially the case for video processing, deep learning engines and numerical code. It is still hard to beat the compiler even in those cases. You have to know about branch prediction, that modern processors execute many instructions in parallel (superscalar) as well as instruction latency. You also need to know about the cache hierarchy, but that's the case even when you let the compiler generate the code. Writing highly optimized code like that is not something every programmer will do, but most will use code written by others that has part in optimized assembly. An example of that is ffmpeg.

PaulDavisThe1st 6y ago
Within Ardour, a cross-platform DAW, we introduced hand-written SSE-based asm for metering, which is the most expensive computational operation in the application (without plugins): compare every single (floating point) audio sample that passes through to some recent max and min, and if larger or smaller, store it as the new max or min. The savings were dramatic, though dependent on the size of the buffers that get processed. With larger buffers, adding this asm reduced DSP load by 30%. This was done 15 years ago, with a few minor updates about 6 years ago. Recently, we've revisited it to add support for AVX on Linux, which was missing. It turns out that gcc can now do more or less as good a job as the hand-crafted asm. The hand-crafted version understood (mis)alignment better than gcc from 15 years ago, but at this point, gcc's inner loop is 1 instruction less than the hand-crafted version, although its setup before reaching the inner loop is several instructions longer. There is still some human skill required to understand the best way to suggest to gcc what is going on, but it increasingly seems as though it's likely best to let the compiler do the actual code generation, even for SIMD.
- compiler-guy 6y ago
  Compilers get better every year. Hand-tuned assembly only gets better if someone pays attention to it. So even if you can beat the compiler today, you might not be able to beat it tomorrow. Therefore it is often useful to have two switchable implementations. High-level code that is basically straightforward, and hand-tuned assembly. You can the compare the two with every new compiler release.
  - gnufx 6y ago
    Not only that, but your micro-architecture-specific hand-tuning probably isn't good for tomorrow's micro-architecture, and definitely not for a new architecture. You still need craft kernels in cases like BLAS, but you really want to reduce them, e.g. BLIS in the case of BLAS.

PaulDavisThe1st 6y ago

Within Ardour, a cross-platform DAW, we introduced hand-written SSE-based asm for metering, which is the most expensive computational operation in the application (without plugins): compare every single (floating point) audio sample that passes through to some recent max and min, and if larger or smaller, store it as the new max or min. The savings were dramatic, though dependent on the size of the buffers that get processed. With larger buffers, adding this asm reduced DSP load by 30%. This was done 15 years ago, with a few minor updates about 6 years ago. Recently, we've revisited it to add support for AVX on Linux, which was missing. It turns out that gcc can now do more or less as good a job as the hand-crafted asm. The hand-crafted version understood (mis)alignment better than gcc from 15 years ago, but at this point, gcc's inner loop is 1 instruction less than the hand-crafted version, although its setup before reaching the inner loop is several instructions longer. There is still some human skill required to understand the best way to suggest to gcc what is going on, but it increasingly seems as though it's likely best to let the compiler do the actual code generation, even for SIMD.

compiler-guy 6y ago
Compilers get better every year. Hand-tuned assembly only gets better if someone pays attention to it. So even if you can beat the compiler today, you might not be able to beat it tomorrow. Therefore it is often useful to have two switchable implementations. High-level code that is basically straightforward, and hand-tuned assembly. You can the compare the two with every new compiler release.
- gnufx 6y ago
  Not only that, but your micro-architecture-specific hand-tuning probably isn't good for tomorrow's micro-architecture, and definitely not for a new architecture. You still need craft kernels in cases like BLAS, but you really want to reduce them, e.g. BLIS in the case of BLAS.