3 ms·

Actually a bit surprised to hear that assembly is faster than optimized C. I figured that compilers are so good nowadays that any gains from hand-written assemb

Actually a bit surprised to hear that assembly is faster than optimized C. I figured that compilers are so good nowadays that any gains from hand-written assembly would be infinitesimal. Clearly I'm wrong on this; I should probably properly learn assembly at some point...

mhh__ 1y ago
Compilers are extremely good considering the amount of crap they have to churn through but they have zero information (by default) about how the program is going to be used so it's not hard to beat them.
- haiku2077 1y ago
  If anyone is curious to learn more, look up "profile-guided optimization" which observes the running program and feeds that information back into the compiler
deleted 1y ago
[deleted]
mananaysiempre 1y ago
Looking at the linked patches, you’ll note that the baseline (ff_detect_range_c) [1] is bog-standard scalar C code while the speedup is achieved in the AVX-512 version (ff_detect_rangeb_avx512) [2] of the same computation. FFmpeg devs prefer to write straight assembly using a library of vector-width-agnostic macros they maintain, but at a glance the equivalent code looks to be straightforwardly expressible in C with Intel intrinsics if that’s more your jam. (Granted, that’s essentially assembly except with a register allocator, so the practical difference is limited.) The vectorization is most of the speedup, not the assembly. To a first approximation, modern compilers can’t vectorize loops beyond the most trivial (say a dot product), and even that you’ll have to ask for (e.g. gcc -O3, which in other cases is often slower than -O2). So for mathy code like this they can easily be a couple dozen times behind in performance compared to wide vectors (AVX/AVX2 or AVX-512), especially when individual elements are small (like the 8-bit ones here). Very tight scalar code, on modern superscalar CPUs... You can outcode a compiler by a meaningful margin, sometimes (my current example is a 40% speedup). But you have to be extremely careful (think dependency chains and execution port loads), and the opportunity does not come often (why are you writing scalar code anyway?..). [1] https://ffmpeg.org/pipermail/ffmpeg-devel/2025-July/346725.html https://ffmpeg.org/pipermail/ffmpeg-devel/2025-July/346725.h... [2] https://ffmpeg.org/pipermail/ffmpeg-devel/2025-July/346726.html https://ffmpeg.org/pipermail/ffmpeg-devel/2025-July/346726.h...
- kasper93 1y ago
  Moreover the baseline _c function is compiled with -march=generic and -fno-tree-vectorize on GCC. Hence it's the best case comparison for handcrafted AVX512 code. And while it's is obviously faster and that's very cool, boasting the 100x may be misinterpreted by outsider readers. I was commenting there with some suggested change and you can find more performance comparison [0]. For example with small adjustment to C and compiling it for AVX512: after (gcc -ftree-vectorize --march=znver4) detect_range_8_c: 285.6 ( 1.00x) detect_range_8_avx2: 256.0 ( 1.12x) detect_range_8_avx512: 107.6 ( 2.65x) Also I argued that it may be a little bit misleading to post comparison without stating the compiler and flags used for said comparison [1]. P.S. There is related work to enable -ftree-vectorize by default [2] [0] https://ffmpeg.org/pipermail/ffmpeg-devel/2025-July/346813.html https://ffmpeg.org/pipermail/ffmpeg-devel/2025-July/346813.h... [1] https://ffmpeg.org/pipermail/ffmpeg-devel/2025-July/346794.html https://ffmpeg.org/pipermail/ffmpeg-devel/2025-July/346794.h... [2] https://ffmpeg.org/pipermail/ffmpeg-devel/2025-July/346439.html https://ffmpeg.org/pipermail/ffmpeg-devel/2025-July/346439.h...
  - etra0 1y ago
    I think this comment should be on the top lol. I mean, I love ffmpeg, I use it a lot and it's fantastic for my needs, but I've found their public persona often misleading and well, this just confirms my bias. > We made a 100x improvement over incredibly unoptimized C by writing heavily specific cpu instructions that the compiler cannot use because we don't allow it! 2x is still an improvement, but way less outstanding as they want it to publicize it because they used assembly.
- nwallin 1y ago
  > the equivalent code looks to be straightforwardly expressible in C with Intel intrinsics if that’s more your jam. (Granted, that’s essentially assembly except with a register allocator, so the practical difference is limited.) The vectorization is most of the speedup, not the assembly. At my day job I have a small pile of code I'm responsible for which is a giant pile of intrinsics. We compile to GCC and MSVC. We have one function that is just a straight function. There are no loops, there is one branch. There is nothing that isn't either a vector intrinsic or an address calculation which I'm pretty sure is simple enough that it can be included in x86's fancy inline memory address calculation thingie. There's basically nothing for the compiler to do except translate vector intrinsics and address calculation from C into assembly. The code when run in GCC is approximately twice as fast as MSVC. The compiler is also responsible for register allocation, and MSVC is criminally insane at it. One of these days I'll have to rewrite this function in assembly, just to get MSVC up to GCC speeds, but frankly I don't want to.
brigade 1y ago
It's AVX512 that makes the gains, not assembly. This kernel is simple enough that it wouldn't be measurably faster than C with AVX512 intrinsics. And it's 100x because a) min/max have single instructions in SIMD vs cmp+cmov in scalar and b) it's operating in u8 precision so each AVX512 instruction does 64x min/max. So unlike the unoptimized scalar that has a throughput under 1 byte per cycle, the AVX512 version can saturate L1 and L2 bandwidth. (128B and 64B per cycle on Zen 5.) But, this kernel is operating on an entire frame; if you have to go to L3 because it's more than a megapixel then the gain should halve (depending on CPU, but assuming Zen 5), and the gain decreases even more if the frame isn't resident in L3.
- saati 1y ago
  The AVX2 version was still 64x faster than the C one, so AVX-512 is just 50% improvement over that. Hand vectorized assembly is very much the key to the gains.
  - brigade 1y ago
    The only material difference AVX2 makes is that it can't saturate L1 bandwidth. Which would imply that 100x for AVX-512 is only for frames that fit within L1. And... yep, the benchmark on 256x16 frames. [1] [1] https://ffmpeg.org/pipermail/ffmpeg-devel/2025-July/346729.html https://ffmpeg.org/pipermail/ffmpeg-devel/2025-July/346729.h...
mafuy 1y ago
If you ever dabble more closely in low level optimization, you will find the first instance of the C compile having a brain fart within less than an hour. Random example: https://stackoverflow.com/questions/71343461/how-does-gcc-not-clang-make-this-optimization-deciding-that-a-store-to-one-str https://stackoverflow.com/questions/71343461/how-does-gcc-no... The code in question was called quadrillions of times, so this actually mattered.
MobiusHorizons 1y ago
Almost all performance critical pieces of c/c++ libraries (including things as seemingly mundane as strlen) use specialized hand written assembly. Compilers are good enough for most people most of the time, but that’s only because most people aren’t writing software that is worth optimizing to this level from a financial perspective.
jesse__ 1y ago
It's extremely easy to beat the compiler by dropping down to SIMD intrinsics. I recently wrote a 4 part .. guide? .. https://scallywag.software/vim/blog/simd-perlin-noise-i https://scallywag.software/vim/blog/simd-perlin-noise-i
asveikau 1y ago
It's SIMD specifically. SIMD is crazy fast and compilers are still not good at generating it.
- globular-toast 1y ago
  There are other things too like using the carry bit directly with ADC instead of using "tricks" to check for overflow before/after it happens for example.

mhh__ 1y ago

Compilers are extremely good considering the amount of crap they have to churn through but they have zero information (by default) about how the program is going to be used so it's not hard to beat them.

haiku2077 1y ago
If anyone is curious to learn more, look up "profile-guided optimization" which observes the running program and feeds that information back into the compiler

deleted 1y ago

[deleted]

mananaysiempre 1y ago

Looking at the linked patches, you’ll note that the baseline (ff_detect_range_c) [1] is bog-standard scalar C code while the speedup is achieved in the AVX-512 version (ff_detect_rangeb_avx512) [2] of the same computation. FFmpeg devs prefer to write straight assembly using a library of vector-width-agnostic macros they maintain, but at a glance the equivalent code looks to be straightforwardly expressible in C with Intel intrinsics if that’s more your jam. (Granted, that’s essentially assembly except with a register allocator, so the practical difference is limited.) The vectorization is most of the speedup, not the assembly. To a first approximation, modern compilers can’t vectorize loops beyond the most trivial (say a dot product), and even that you’ll have to ask for (e.g. gcc -O3, which in other cases is often slower than -O2). So for mathy code like this they can easily be a couple dozen times behind in performance compared to wide vectors (AVX/AVX2 or AVX-512), especially when individual elements are small (like the 8-bit ones here). Very tight scalar code, on modern superscalar CPUs... You can outcode a compiler by a meaningful margin, sometimes (my current example is a 40% speedup). But you have to be extremely careful (think dependency chains and execution port loads), and the opportunity does not come often (why are you writing scalar code anyway?..). [1] https://ffmpeg.org/pipermail/ffmpeg-devel/2025-July/346725.html https://ffmpeg.org/pipermail/ffmpeg-devel/2025-July/346725.h... [2] https://ffmpeg.org/pipermail/ffmpeg-devel/2025-July/346726.html https://ffmpeg.org/pipermail/ffmpeg-devel/2025-July/346726.h...

kasper93 1y ago
Moreover the baseline _c function is compiled with -march=generic and -fno-tree-vectorize on GCC. Hence it's the best case comparison for handcrafted AVX512 code. And while it's is obviously faster and that's very cool, boasting the 100x may be misinterpreted by outsider readers. I was commenting there with some suggested change and you can find more performance comparison [0]. For example with small adjustment to C and compiling it for AVX512: after (gcc -ftree-vectorize --march=znver4) detect_range_8_c: 285.6 ( 1.00x) detect_range_8_avx2: 256.0 ( 1.12x) detect_range_8_avx512: 107.6 ( 2.65x) Also I argued that it may be a little bit misleading to post comparison without stating the compiler and flags used for said comparison [1]. P.S. There is related work to enable -ftree-vectorize by default [2] [0] https://ffmpeg.org/pipermail/ffmpeg-devel/2025-July/346813.html https://ffmpeg.org/pipermail/ffmpeg-devel/2025-July/346813.h... [1] https://ffmpeg.org/pipermail/ffmpeg-devel/2025-July/346794.html https://ffmpeg.org/pipermail/ffmpeg-devel/2025-July/346794.h... [2] https://ffmpeg.org/pipermail/ffmpeg-devel/2025-July/346439.html https://ffmpeg.org/pipermail/ffmpeg-devel/2025-July/346439.h...
- etra0 1y ago
  I think this comment should be on the top lol. I mean, I love ffmpeg, I use it a lot and it's fantastic for my needs, but I've found their public persona often misleading and well, this just confirms my bias. > We made a 100x improvement over incredibly unoptimized C by writing heavily specific cpu instructions that the compiler cannot use because we don't allow it! 2x is still an improvement, but way less outstanding as they want it to publicize it because they used assembly.
nwallin 1y ago
> the equivalent code looks to be straightforwardly expressible in C with Intel intrinsics if that’s more your jam. (Granted, that’s essentially assembly except with a register allocator, so the practical difference is limited.) The vectorization is most of the speedup, not the assembly. At my day job I have a small pile of code I'm responsible for which is a giant pile of intrinsics. We compile to GCC and MSVC. We have one function that is just a straight function. There are no loops, there is one branch. There is nothing that isn't either a vector intrinsic or an address calculation which I'm pretty sure is simple enough that it can be included in x86's fancy inline memory address calculation thingie. There's basically nothing for the compiler to do except translate vector intrinsics and address calculation from C into assembly. The code when run in GCC is approximately twice as fast as MSVC. The compiler is also responsible for register allocation, and MSVC is criminally insane at it. One of these days I'll have to rewrite this function in assembly, just to get MSVC up to GCC speeds, but frankly I don't want to.

brigade 1y ago

It's AVX512 that makes the gains, not assembly. This kernel is simple enough that it wouldn't be measurably faster than C with AVX512 intrinsics. And it's 100x because a) min/max have single instructions in SIMD vs cmp+cmov in scalar and b) it's operating in u8 precision so each AVX512 instruction does 64x min/max. So unlike the unoptimized scalar that has a throughput under 1 byte per cycle, the AVX512 version can saturate L1 and L2 bandwidth. (128B and 64B per cycle on Zen 5.) But, this kernel is operating on an entire frame; if you have to go to L3 because it's more than a megapixel then the gain should halve (depending on CPU, but assuming Zen 5), and the gain decreases even more if the frame isn't resident in L3.

saati 1y ago
The AVX2 version was still 64x faster than the C one, so AVX-512 is just 50% improvement over that. Hand vectorized assembly is very much the key to the gains.
- brigade 1y ago
  The only material difference AVX2 makes is that it can't saturate L1 bandwidth. Which would imply that 100x for AVX-512 is only for frames that fit within L1. And... yep, the benchmark on 256x16 frames. [1] [1] https://ffmpeg.org/pipermail/ffmpeg-devel/2025-July/346729.html https://ffmpeg.org/pipermail/ffmpeg-devel/2025-July/346729.h...

mafuy 1y ago

If you ever dabble more closely in low level optimization, you will find the first instance of the C compile having a brain fart within less than an hour. Random example: https://stackoverflow.com/questions/71343461/how-does-gcc-not-clang-make-this-optimization-deciding-that-a-store-to-one-str https://stackoverflow.com/questions/71343461/how-does-gcc-no... The code in question was called quadrillions of times, so this actually mattered.

MobiusHorizons 1y ago

Almost all performance critical pieces of c/c++ libraries (including things as seemingly mundane as strlen) use specialized hand written assembly. Compilers are good enough for most people most of the time, but that’s only because most people aren’t writing software that is worth optimizing to this level from a financial perspective.

jesse__ 1y ago

It's extremely easy to beat the compiler by dropping down to SIMD intrinsics. I recently wrote a 4 part .. guide? .. https://scallywag.software/vim/blog/simd-perlin-noise-i https://scallywag.software/vim/blog/simd-perlin-noise-i

asveikau 1y ago

It's SIMD specifically. SIMD is crazy fast and compilers are still not good at generating it.

globular-toast 1y ago
There are other things too like using the carry bit directly with ADC instead of using "tricks" to check for overflow before/after it happens for example.