6 ms·

When I spent a decade doing SIMD optimizations for HEVC (among other things), it was sort of a joke to compare the assembly versions to plain c. Because you’d g

When I spent a decade doing SIMD optimizations for HEVC (among other things), it was sort of a joke to compare the assembly versions to plain c. Because you’d get some ridiculous multipliers like 100x. It is pretty misleading, what it really means is it was extremely inefficient to begin with. The devil is in the details, microbenchmarks are typically calling the same function a million times in a loop and everything gets cached reducing the overhead to sheer cpu cycles. But that’s not how it’s actually used in the wild. It might be called once in a sea of many many other things. You can at least go out of your way to create a massive test region of memory to prevent the cache from being so hot, but I doubt they do that.

torginus 1y ago
Sorry for the derail, but it sounds like you have a ton of experience with SIMD. Have you used ISPC, and what are your thoughts on it? I feel it's a bit ridiculous that in this day and age you have to write SIMD code by hand, as regular compilers suck at auto-vectorizing, especially as this has never been the case with GPU kernels.
- almostgotcaught 1y ago
  > Have you used ISPC No professional kernel writer uses Auto-vectorization. > I feel it's a bit ridiculous that in this day and age you have to write SIMD code by hand You feel it's ridiculous because you've been sold a myth/lie (abstraction). In reality the details have always mattered.
  - CyberDildonics 1y ago
    ISPC is a lot different from C++ compiler auto vectorization and it works extremely well. Have you tried it or not? If so where does it actually fall down? It warns you when doing slow stuff like gathers and scatters.
    - camel-cdr 1y ago
      Is it a lot different from autovec with #pragma omp simd? I played around with ISPC a bit and it didn't seem much different.
      - CyberDildonics 1y ago
        I don't have any experience with that aspect of openmp. When you use ISPC are you using the varying and uniform keywords? You can write something that is almost C but it basically forced to vectorize. If you both are vectorizing the same thing there might not be much difference. If both are not vectorizing there might not be much difference, but with ISPC you can easily make sure that it does use vectorization and the best instruction set for your CPU.
  - rerdavies 1y ago
    I'm hard pressed to think of a kernel function that would benefit from auto-vectorization.
    - aa-jv 1y ago
      A good example is a vector addition kernel, which is simple, embarrassingly parallel, and well-suited for SIMD: void vector_add(float *a, float *b, float *c, int n) { for (int i = 0; i < n; i++) { c[i] = a[i] + b[i]; } }
      - rerdavies 1y ago
        OK. And which kernel function does that? Serious question.
        rerdavies 1y ago
        Ok. Miscommunication. Different kernels. I was actually asking "which Linux kernel functions are going to benefit from auto-vectorization. There are not a lot of Linux kernel functions that take arguments that are two arrays of floating point values. Lots of string stuff that typically doesn't vectorize well (except in freakish cases), perhaps. But very very few functions that take arrays of floats as arguments. Yes, graphics libraries if you want to count those as kernel functions; but those primarily concern themselves with passing arrays of floats to co-processors, to be vectorized by a GPU (different problem). As an interesting point of reference, the last time I did Windows kernel development (which was admittedly not recently), code running in ring 0 was not allowed to access SIMD registers because they weren't saved and loaded during kernel-code context switches. Not sure if that's the case, but it probably is. Context switches are a LOT faster if you don't have to save and load SIMD registers.
      - Const-me 1y ago
        Indeed, automatic vectorizers do such simple things pretty reliably these days. However, if you build your software from kernels like that, you leave a lot of performance on the table. For example, each core of my Zen 4 CPU at base frequency can add FP32 numbers with AVX1 or AVX-512 at 268 GB/sec, which results in 806 GB/sec total bandwidth for your kernel with two inputs and 1 output. However, dual-channel DDR5 memory in my computer can only deliver 83 GB/sec bandwidth shared across all CPU cores. That’s an order of magnitude difference for a single threaded program, and almost 2 orders of magnitude difference when computing something on the complete CPU. Even worse, the difference between compute and memory widens over time. The next generation Zen 5 CPUs can add numbers twice as fast per cycle if using AVX-512. For this reason, ideally you want your kernels to do much more work with the numbers loaded from memory. That’s why efficient compute kernels are often way more complicated than the for loop in your example. Sadly, seems modern compilers can only reliably autovectorize very simple loops.
- capyba 1y ago
  Personally I’ve never been able to beat gcc or icx autovectorization by using intrinsics; often I’m slower by a factor of 1.5-2x. Do you have any wisdom you can share about techniques or references you can point to?
  - jesse__ 1y ago
    I recently finished a 4 part series about vectorizing perlin noise.. from the very basics up to beating the state-of-the-art by 1.8x https://scallywag.software/vim/blog/simd-perlin-noise-i https://scallywag.software/vim/blog/simd-perlin-noise-i
    - capyba 1y ago
      Very cool, thank you for sharing!
  - mort96 1y ago
    You may not be able to beat the auto-vectorizer for problems which can be auto-vectorized, but you have almost certainly encountered situations which the compiler can't auto-vectorize but where you could've manually written a SIMD implementation which beats the compiler's scalar code.
    - capyba 1y ago
      Excellent point, that makes a lot of sense. You’re absolutely right that I’ve only tried problems that could be auto vectorized…
- rerdavies 1y ago
  Regular compilers are actually extraordinarily good at auto-vectorizing. There are a few oddities that one has to be aware of; but in my experience, if you offer a GCC or Clang compiler an opportunity to auto-vectorize, it will leap on it pretty ruthlessly. I have done a lot of work recently with auto-vectorizing code for high-performance realtime audio processing. And it's pretty extraordinary how good GCC and Clang are. (MSVC is allegedly equally good, but I don't have recent experience with it). And they will generally produce better assembler than I can for all but the weird bit-twiddly Intel special-purpose SIMD instructions. I'm actually professionally good at it. But the GCC instruction scheduler seems to be consistently better than I am at instruction scheduling. GCC and Clang actually have detailed models of the execution pipelines of all x64 and aarch64 processors that are used to perform instruction scheduling . Truly an amazing piece of work. So their instruction scheduling is -- as far as I can tell without very expensive Intel and ARM profiling tools -- immaculate across a wide variety of processors and architectures. And if it vectorizes on x64, it will probably vectorize equally well on aarch64 ARM Neon as well. And if you want optimal instruction scheduling for an Arm Cortex A72 processor (a pi 4), well, there's a switch for that. And for an A76 (a Pi 5). And for an Intel N100. The basics: - You need --fastmath -O3 (or the MSVC eqivalent). - You need to use and understand the "restrict" keyword (or closest compiler equivalent). If the compiler has to guard against aliasing of input and output arrays, you'll get performance-impaired code. - You can reasonably expect any math in a for loop to be vectorized provided there aren't dependencies between iterations of the loop. - Operations on arrays and matrices whose size is fixed at compile time are significantly better than operations that operate on matrices and arrays whose size is determined at runtime. (Dealing with the raggedy non-x4 tail end of arrays is easier if the shape is known at compile time). - trust the compiler. It will do some pretty extraordinary things. (e.g. generating 7 bit-twiddly SIMD instructions to vectorize atanf(float4). But verify. Make sure it is doing what you expect. - the cycle is: compile the C/C++ code; disassemble to make sure it did vectorize properly (and that it doesn't have checks for aliased pointers); profile; repeat until done, or forever, whichever comes first. - Even for fairly significant compute, execution is likely to be dominated by memory reads and writes (hopefully cached memory reads and writes). My Neural net code spends about 80% of its time waiting for reads and writes to various cache levels. (And 15% of its time doing properly vectorized atanf operations, some significant portion of which involves memory reads and writes that spill L1 cache). The FFT code spends pretty close to 100% of its time waiting for memory reads and writes (even with pivots that improve cache behavior). The actual optimization effort was determing (at compile time) when to do pivot rounds to improve use of L1 and L2 cache. I would expect this to be generally true. You can do a lot of compute in the time it takes for an L1 cache miss to execute. I can't think of why ISPC would do better than GCC,given what GCC actually does. My suspicion is that ISPC was a technology demonstration rather than a real product produced at a time when major compilers had no support at all for SIMD.
- jandrewrogers 1y ago
  The reason you have to optimize SIMD by hand is that compilers can't redesign your data structures and algorithms. This is the level of abstraction at which you often need to be working with SIMD. Compilers are limited to codegen things like auto-vectorizing simple loops, but that isn't where most of the interesting possibilities are with SIMD. If you look at heavily-optimized SIMD code side-by-side with the equivalent heavily-optimized scalar code, they are often almost entirely unrelated implementations. That's the part compilers can't do. Note that I use SIMD heavily in a lot of domains that aren't just brute-forcing a lot straightforward numerics. If you are just brute-forcing numerics, that auto-vectorization works pretty well. For example, I have a high-performance I/O scheduler that is almost pure AVX-512 and the compiler can't vectorize any of that.
  - lenkite 1y ago
    Hope you one day can get around to writing some blog posts or maybe even publish some books/courses on this - it sounds interesting!
    - dwood_dev 1y ago
      You'll find the video in this submission a fun watch if you want to learn a bit about optimization and AVX-512. https://news.ycombinator.com/item?id=44176729 https://news.ycombinator.com/item?id=44176729
- brandmeyer 1y ago
  ISPC suffers from poor scatter and gather support in hardware. The direct result is that it is hard to make programs that scale in complexity without resorting to shenanigans. An ideal scatter-read or gather-store instruction should take time proportional to the number of cache lines that it touches. If all of the lane accesses are sequential and cache line aligned it should take the same amount of time as an aligned vector load or store. If the accesses have high cache locality such that only two cache lines are touched, it should cost exactly the same as loading those two cache lines and shuffling the results into place. That isn't what we have on x86-AVX512. They are microcoded with inefficient lane-at-a-time implementations. If you know that there is good locality of reference in the access, then it can be faster to hand-code your own cache line-at-a-time load/shuffle/masked-merge loop than to rely on the hardware. This makes me sad. ISPC's varying variables have no way to declare that they are sequential among all lanes. Therefore, without extensive inlining to expose the caller's access pattern, it issues scatters and gathers at the drop of a hat. You might like to write your program with a naive x[y] (x a uniform pointer, y a varying index) in a subroutine, but ISPC's language cannot infer that y is sequential along lanes. So, you have to carefully re-code it to say that y is actually a uniform offset into the array, and write x[y + programIndex]. Error-prone, yet utterly essential for decent performance. I resorted to munging my naming conventions for such indexes, not unlike the Hungarian notation of yesteryear. Rewriting critical data structures in SoA format instead of AoS format is non-trivial, and a prerequisite to get decent performance from ISPC. You cannot "just" replace some subroutines with ISPC routines, you need to make major refactorings that touch the rest of the program as well. This is neutral in an ISPC-versus-intrinsics (or even ISPC-versus-GPU) shootout, but it is worth mentioning only to point out that ISPC is also not a silver bullet in this regards, either. Non-minor nit: The ISPC math library gives up far too much precision by default in the name of speed. Fortunately, Sleef is not terribly difficult to integrate and use for the 1-ulp max rounding error that I've come to expect from a competent libm. Another: The ISPC calling convention adheres rather strictly to the C calling convention... which doesn't provide any callee-saved vector registers, not even for the execution mask. So if you like to decompose your program across multiple compilation units, you will also notice much more register save and restore traffic than you would like or expect. I want to like it, I can get some work done in it, and I did get significant performance improvements over scalar code when using it. But the resulting source code and object code are not great. They are merely acceptable.
izabera 1y ago
ffmpeg is not too different from a microbenchmark, the whole program is basically just: while (read(buf)) write(transform(buf))
- fuzztester 1y ago
  the devil is in the details (of the holy assembly). thus sayeth the lord. praise the lord!
- vlovich123 1y ago
  > However, the developers were soon to clarify that the 100x claim applies to just a single function, “not the whole of FFmpeg.” So OP is correct. The 100x speed up is according to some misleading micro benchmark. The reason is that that transform is a huge amount of code and as OP said this will blow out the code cache while the amount of data you’re processing results in a blowout of the data cache. Net overall improvement might be 1% if even that.
  - majewsky 1y ago
    > The 100x speed up is according to some misleading micro benchmark. Honestly though, nobody who has any idea how anything works would have expected ffmpeg to suddenly unearth a 100x speedup for everything. That's why the devs did not clarify this right away. It's too laughable of an assumption.
  - rerdavies 1y ago
    And it's probable that the developer is comparing code compiled at -O0 (no optimization) against hand-coded assembler, like they did the last time they claimed a 90x speed up. So just to to summary: either a 100x, or a 100% speedup (depending on which source) - comparing hand-coded assembler vs. unoptimized C code. - on a function that was poorly written in the first place. - in code that's so rarely used that nobody could be bothered to fix it for decades. - and even then, a tiny function whose overall CPU cost was about 2% of CPU cost to perform the obsolete task that nobody cared about enough to fix. - so basically code that fails the profile before optimize rule, and should never have been optimized in the first place. I think that covers it.
yieldcrv 1y ago
> what it really means is it was extremely inefficient to begin with I care more about the outcome than the underlying semantics, to me thats kind of a given
- dylan604 1y ago
  Any average user would be more concerned with the end results. But this is a forum of not average users and more the people specifically that get off on the underlying semantics. Just look at how many people here are so infatuated with AI/LLMs and are so concerned about training data, models, number of tokens, yet completely gloss over the fact that every single one of these products will just make shit up. Those concerned with the underlying semantics seem to not give a damn about the outcome.
  - rerdavies 1y ago
    Which seems an extremely odd point to make in a thread about a story that concerns itself with humans just making shit up.
bee_rider 1y ago
Sadly, even beyond the hot cache issue, > They would later go on to elaborate that the functionality, which might enjoy a 100% speed boost, depending upon your system, was “an obscure filter.” However, to be fair, they communicate this stuff very clearly.
Cthulhu_ 1y ago
Does FFmpeg and co have "macrobenchmarks" as well? I would imagine software like that would have a diverse set of videos and a bajillion different encoding / decoding / transformation sets that are used to measure performance (time, cpu, file size, quality) over time. But it would need dedicated and consistent hardware to test all of that.
- eru 1y ago
  You don't actually need neither dedicated nor consistent hardware, if you are willing to do some statistics. Basically, you'd do a block design (https://en.wikipedia.org/wiki/Blocking_(statistics) https://en.wikipedia.org/wiki/Blocking_(statistics)): on any random hardware you have, you run both versions back to back (or even better, interleaved), and note down the performance. The idea that the differences in machines themselves and anything else running on them are noise, and you are trying to design your experiments in such a way that the noise should affect arms of the experiment in the same way---at least statistically. Downsides: you have to do more runs and do more statistics to deal with the noise. Upside: you can use any old hardware you have access to, even if it's not dedicated. And the numbers are arguably going to be more representative of real conditions, and not just a pristine lab environment.

torginus 1y ago

Sorry for the derail, but it sounds like you have a ton of experience with SIMD. Have you used ISPC, and what are your thoughts on it? I feel it's a bit ridiculous that in this day and age you have to write SIMD code by hand, as regular compilers suck at auto-vectorizing, especially as this has never been the case with GPU kernels.

almostgotcaught 1y ago
> Have you used ISPC No professional kernel writer uses Auto-vectorization. > I feel it's a bit ridiculous that in this day and age you have to write SIMD code by hand You feel it's ridiculous because you've been sold a myth/lie (abstraction). In reality the details have always mattered.
- CyberDildonics 1y ago
  ISPC is a lot different from C++ compiler auto vectorization and it works extremely well. Have you tried it or not? If so where does it actually fall down? It warns you when doing slow stuff like gathers and scatters.
  - camel-cdr 1y ago
    Is it a lot different from autovec with #pragma omp simd? I played around with ISPC a bit and it didn't seem much different.
    - CyberDildonics 1y ago
      I don't have any experience with that aspect of openmp. When you use ISPC are you using the varying and uniform keywords? You can write something that is almost C but it basically forced to vectorize. If you both are vectorizing the same thing there might not be much difference. If both are not vectorizing there might not be much difference, but with ISPC you can easily make sure that it does use vectorization and the best instruction set for your CPU.
- rerdavies 1y ago
  I'm hard pressed to think of a kernel function that would benefit from auto-vectorization.
  - aa-jv 1y ago
    A good example is a vector addition kernel, which is simple, embarrassingly parallel, and well-suited for SIMD: void vector_add(float *a, float *b, float *c, int n) { for (int i = 0; i < n; i++) { c[i] = a[i] + b[i]; } }
    - rerdavies 1y ago
      OK. And which kernel function does that? Serious question.
      - rerdavies 1y ago
        Ok. Miscommunication. Different kernels. I was actually asking "which Linux kernel functions are going to benefit from auto-vectorization. There are not a lot of Linux kernel functions that take arguments that are two arrays of floating point values. Lots of string stuff that typically doesn't vectorize well (except in freakish cases), perhaps. But very very few functions that take arrays of floats as arguments. Yes, graphics libraries if you want to count those as kernel functions; but those primarily concern themselves with passing arrays of floats to co-processors, to be vectorized by a GPU (different problem). As an interesting point of reference, the last time I did Windows kernel development (which was admittedly not recently), code running in ring 0 was not allowed to access SIMD registers because they weren't saved and loaded during kernel-code context switches. Not sure if that's the case, but it probably is. Context switches are a LOT faster if you don't have to save and load SIMD registers.
    - Const-me 1y ago
      Indeed, automatic vectorizers do such simple things pretty reliably these days. However, if you build your software from kernels like that, you leave a lot of performance on the table. For example, each core of my Zen 4 CPU at base frequency can add FP32 numbers with AVX1 or AVX-512 at 268 GB/sec, which results in 806 GB/sec total bandwidth for your kernel with two inputs and 1 output. However, dual-channel DDR5 memory in my computer can only deliver 83 GB/sec bandwidth shared across all CPU cores. That’s an order of magnitude difference for a single threaded program, and almost 2 orders of magnitude difference when computing something on the complete CPU. Even worse, the difference between compute and memory widens over time. The next generation Zen 5 CPUs can add numbers twice as fast per cycle if using AVX-512. For this reason, ideally you want your kernels to do much more work with the numbers loaded from memory. That’s why efficient compute kernels are often way more complicated than the for loop in your example. Sadly, seems modern compilers can only reliably autovectorize very simple loops.
capyba 1y ago
Personally I’ve never been able to beat gcc or icx autovectorization by using intrinsics; often I’m slower by a factor of 1.5-2x. Do you have any wisdom you can share about techniques or references you can point to?
- jesse__ 1y ago
  I recently finished a 4 part series about vectorizing perlin noise.. from the very basics up to beating the state-of-the-art by 1.8x https://scallywag.software/vim/blog/simd-perlin-noise-i https://scallywag.software/vim/blog/simd-perlin-noise-i
  - capyba 1y ago
    Very cool, thank you for sharing!
- mort96 1y ago
  You may not be able to beat the auto-vectorizer for problems which can be auto-vectorized, but you have almost certainly encountered situations which the compiler can't auto-vectorize but where you could've manually written a SIMD implementation which beats the compiler's scalar code.
  - capyba 1y ago
    Excellent point, that makes a lot of sense. You’re absolutely right that I’ve only tried problems that could be auto vectorized…
rerdavies 1y ago
Regular compilers are actually extraordinarily good at auto-vectorizing. There are a few oddities that one has to be aware of; but in my experience, if you offer a GCC or Clang compiler an opportunity to auto-vectorize, it will leap on it pretty ruthlessly. I have done a lot of work recently with auto-vectorizing code for high-performance realtime audio processing. And it's pretty extraordinary how good GCC and Clang are. (MSVC is allegedly equally good, but I don't have recent experience with it). And they will generally produce better assembler than I can for all but the weird bit-twiddly Intel special-purpose SIMD instructions. I'm actually professionally good at it. But the GCC instruction scheduler seems to be consistently better than I am at instruction scheduling. GCC and Clang actually have detailed models of the execution pipelines of all x64 and aarch64 processors that are used to perform instruction scheduling . Truly an amazing piece of work. So their instruction scheduling is -- as far as I can tell without very expensive Intel and ARM profiling tools -- immaculate across a wide variety of processors and architectures. And if it vectorizes on x64, it will probably vectorize equally well on aarch64 ARM Neon as well. And if you want optimal instruction scheduling for an Arm Cortex A72 processor (a pi 4), well, there's a switch for that. And for an A76 (a Pi 5). And for an Intel N100. The basics: - You need --fastmath -O3 (or the MSVC eqivalent). - You need to use and understand the "restrict" keyword (or closest compiler equivalent). If the compiler has to guard against aliasing of input and output arrays, you'll get performance-impaired code. - You can reasonably expect any math in a for loop to be vectorized provided there aren't dependencies between iterations of the loop. - Operations on arrays and matrices whose size is fixed at compile time are significantly better than operations that operate on matrices and arrays whose size is determined at runtime. (Dealing with the raggedy non-x4 tail end of arrays is easier if the shape is known at compile time). - trust the compiler. It will do some pretty extraordinary things. (e.g. generating 7 bit-twiddly SIMD instructions to vectorize atanf(float4). But verify. Make sure it is doing what you expect. - the cycle is: compile the C/C++ code; disassemble to make sure it did vectorize properly (and that it doesn't have checks for aliased pointers); profile; repeat until done, or forever, whichever comes first. - Even for fairly significant compute, execution is likely to be dominated by memory reads and writes (hopefully cached memory reads and writes). My Neural net code spends about 80% of its time waiting for reads and writes to various cache levels. (And 15% of its time doing properly vectorized atanf operations, some significant portion of which involves memory reads and writes that spill L1 cache). The FFT code spends pretty close to 100% of its time waiting for memory reads and writes (even with pivots that improve cache behavior). The actual optimization effort was determing (at compile time) when to do pivot rounds to improve use of L1 and L2 cache. I would expect this to be generally true. You can do a lot of compute in the time it takes for an L1 cache miss to execute. I can't think of why ISPC would do better than GCC,given what GCC actually does. My suspicion is that ISPC was a technology demonstration rather than a real product produced at a time when major compilers had no support at all for SIMD.
jandrewrogers 1y ago
The reason you have to optimize SIMD by hand is that compilers can't redesign your data structures and algorithms. This is the level of abstraction at which you often need to be working with SIMD. Compilers are limited to codegen things like auto-vectorizing simple loops, but that isn't where most of the interesting possibilities are with SIMD. If you look at heavily-optimized SIMD code side-by-side with the equivalent heavily-optimized scalar code, they are often almost entirely unrelated implementations. That's the part compilers can't do. Note that I use SIMD heavily in a lot of domains that aren't just brute-forcing a lot straightforward numerics. If you are just brute-forcing numerics, that auto-vectorization works pretty well. For example, I have a high-performance I/O scheduler that is almost pure AVX-512 and the compiler can't vectorize any of that.
- lenkite 1y ago
  Hope you one day can get around to writing some blog posts or maybe even publish some books/courses on this - it sounds interesting!
  - dwood_dev 1y ago
    You'll find the video in this submission a fun watch if you want to learn a bit about optimization and AVX-512. https://news.ycombinator.com/item?id=44176729 https://news.ycombinator.com/item?id=44176729
brandmeyer 1y ago
ISPC suffers from poor scatter and gather support in hardware. The direct result is that it is hard to make programs that scale in complexity without resorting to shenanigans. An ideal scatter-read or gather-store instruction should take time proportional to the number of cache lines that it touches. If all of the lane accesses are sequential and cache line aligned it should take the same amount of time as an aligned vector load or store. If the accesses have high cache locality such that only two cache lines are touched, it should cost exactly the same as loading those two cache lines and shuffling the results into place. That isn't what we have on x86-AVX512. They are microcoded with inefficient lane-at-a-time implementations. If you know that there is good locality of reference in the access, then it can be faster to hand-code your own cache line-at-a-time load/shuffle/masked-merge loop than to rely on the hardware. This makes me sad. ISPC's varying variables have no way to declare that they are sequential among all lanes. Therefore, without extensive inlining to expose the caller's access pattern, it issues scatters and gathers at the drop of a hat. You might like to write your program with a naive x[y] (x a uniform pointer, y a varying index) in a subroutine, but ISPC's language cannot infer that y is sequential along lanes. So, you have to carefully re-code it to say that y is actually a uniform offset into the array, and write x[y + programIndex]. Error-prone, yet utterly essential for decent performance. I resorted to munging my naming conventions for such indexes, not unlike the Hungarian notation of yesteryear. Rewriting critical data structures in SoA format instead of AoS format is non-trivial, and a prerequisite to get decent performance from ISPC. You cannot "just" replace some subroutines with ISPC routines, you need to make major refactorings that touch the rest of the program as well. This is neutral in an ISPC-versus-intrinsics (or even ISPC-versus-GPU) shootout, but it is worth mentioning only to point out that ISPC is also not a silver bullet in this regards, either. Non-minor nit: The ISPC math library gives up far too much precision by default in the name of speed. Fortunately, Sleef is not terribly difficult to integrate and use for the 1-ulp max rounding error that I've come to expect from a competent libm. Another: The ISPC calling convention adheres rather strictly to the C calling convention... which doesn't provide any callee-saved vector registers, not even for the execution mask. So if you like to decompose your program across multiple compilation units, you will also notice much more register save and restore traffic than you would like or expect. I want to like it, I can get some work done in it, and I did get significant performance improvements over scalar code when using it. But the resulting source code and object code are not great. They are merely acceptable.

izabera 1y ago

ffmpeg is not too different from a microbenchmark, the whole program is basically just: while (read(buf)) write(transform(buf))

fuzztester 1y ago
the devil is in the details (of the holy assembly). thus sayeth the lord. praise the lord!
vlovich123 1y ago
> However, the developers were soon to clarify that the 100x claim applies to just a single function, “not the whole of FFmpeg.” So OP is correct. The 100x speed up is according to some misleading micro benchmark. The reason is that that transform is a huge amount of code and as OP said this will blow out the code cache while the amount of data you’re processing results in a blowout of the data cache. Net overall improvement might be 1% if even that.
- majewsky 1y ago
  > The 100x speed up is according to some misleading micro benchmark. Honestly though, nobody who has any idea how anything works would have expected ffmpeg to suddenly unearth a 100x speedup for everything. That's why the devs did not clarify this right away. It's too laughable of an assumption.
- rerdavies 1y ago
  And it's probable that the developer is comparing code compiled at -O0 (no optimization) against hand-coded assembler, like they did the last time they claimed a 90x speed up. So just to to summary: either a 100x, or a 100% speedup (depending on which source) - comparing hand-coded assembler vs. unoptimized C code. - on a function that was poorly written in the first place. - in code that's so rarely used that nobody could be bothered to fix it for decades. - and even then, a tiny function whose overall CPU cost was about 2% of CPU cost to perform the obsolete task that nobody cared about enough to fix. - so basically code that fails the profile before optimize rule, and should never have been optimized in the first place. I think that covers it.

yieldcrv 1y ago

> what it really means is it was extremely inefficient to begin with I care more about the outcome than the underlying semantics, to me thats kind of a given

dylan604 1y ago
Any average user would be more concerned with the end results. But this is a forum of not average users and more the people specifically that get off on the underlying semantics. Just look at how many people here are so infatuated with AI/LLMs and are so concerned about training data, models, number of tokens, yet completely gloss over the fact that every single one of these products will just make shit up. Those concerned with the underlying semantics seem to not give a damn about the outcome.
- rerdavies 1y ago
  Which seems an extremely odd point to make in a thread about a story that concerns itself with humans just making shit up.

bee_rider 1y ago

Sadly, even beyond the hot cache issue, > They would later go on to elaborate that the functionality, which might enjoy a 100% speed boost, depending upon your system, was “an obscure filter.” However, to be fair, they communicate this stuff very clearly.

Cthulhu_ 1y ago

Does FFmpeg and co have "macrobenchmarks" as well? I would imagine software like that would have a diverse set of videos and a bajillion different encoding / decoding / transformation sets that are used to measure performance (time, cpu, file size, quality) over time. But it would need dedicated and consistent hardware to test all of that.

eru 1y ago
You don't actually need neither dedicated nor consistent hardware, if you are willing to do some statistics. Basically, you'd do a block design (https://en.wikipedia.org/wiki/Blocking_(statistics) https://en.wikipedia.org/wiki/Blocking_(statistics)): on any random hardware you have, you run both versions back to back (or even better, interleaved), and note down the performance. The idea that the differences in machines themselves and anything else running on them are noise, and you are trying to design your experiments in such a way that the noise should affect arms of the experiment in the same way---at least statistically. Downsides: you have to do more runs and do more statistics to deal with the noise. Upside: you can use any old hardware you have access to, even if it's not dedicated. And the numbers are arguably going to be more representative of real conditions, and not just a pristine lab environment.