3 ms·

After years of staring at HFT disassembly, I'm thoroughly disappointed by autovectorization. Here's the simplest godbolt example I could think of to illustrate

After years of staring at HFT disassembly, I'm thoroughly disappointed by autovectorization. Here's the simplest godbolt example I could think of to illustrate, summing a string of fixed length and non-fixed length: https://godbolt.org/z/O_M5fU https://godbolt.org/z/O_M5fU You can see that the most recent GCC fails to use the AVX512 zmm registers even after being configured to do so (afaik) and also fails to use more than 4 registers. Clang does better, using zmm* and all the registers. But in both cases, the amount of code generated is quite large. If you compile with -Os instead of -O3, no vector instructions are used for some reason. So when you load this code, no matter what, you're loading a bunch of instruction cachelines, which will destroy most of your latency gain unless the input is very large. And even if your input is large, you'll fault that data anyways. So what's the point of doing this on the CPU again?

m0zg 8y ago
Staring is not enough, you need to benchmark. You might be surprised. The results are often counterintuitive, because it's hard for a human to account for how the program _actually_ gets executed, which instructions go in which order, port utilization, data dependencies, CPU bugs, effects of alignment (or, often, lack thereof on Intel CPUs), micro-op caching, etc etc. For all you know GCC deliberately avoided the use of AVX512 registers because AVX512 causes the CPU to throttle the clock. I've also found that ye olde -O2 flag recommendation which says -O3 is likely to be slower in practice because it produces bulkier machine code, nearly never holds up anymore. That said, I concede that autovectorization is not perfect. I can almost always beat it if I really want to, sometimes by quite a margin. But what you're proposing is unlikely to help matters, especially for smaller inputs that you're concerned about.
- CoolGuySteve 8y ago
  I have benchmarked similar code extensively, this is a toy version of a FIX checksum. It's not "counterintuitive", it's just broken, especially for infrequently executed code where the instruction fetch stalls will kill your latency. You are welcomed to load up the godbolt example and try and create better assembly using O2 or whatever else you can think of. You can build the code on your machine and benchmark it too. But at the end of the day, the compiler output sucks. It's clearly optimized for synthetic benchmarks that chug through large datasets using a tiny amount of code that can fit in L1 cache even after the compiler blows up its footprint. But my point is that these large datasets with tiny processing kernels (like a video codec for example) are much better suited to different kinds of processors.
dragontamer 8y ago
I don't believe Knights Landing supports an 8-bit ZMM add. Changing the code to 32-bit ints results in the "vpaddd zmm0, zmm0, ZMMWORD PTR [rdx]" that you'd expect from the auto-vectorizer. > You can see that the most recent GCC fails to use the AVX512 zmm registers even after being configured to do so (afaik) and also fails to use more than 4 registers In the 32-bit vpaddd code... "vpaddd zmm0, zmm0, ZMMWORD PTR [rdx]" becomes a new zmm register allocation in the register-renamer (due to a cut dependency). I doubt any code would be any faster. In any case, its not about "the number of registers used", proper analysis is about the depth of dependencies. I'd only be worried about small-register usage if the dependency chain is large (which doesn't seem to be the case). EDIT: I had some bad analysis. I've erased the bad paragraph. So it seems pretty good in my eyes.
- CoolGuySteve 8y ago
  The clang code loads the 8-bit words into 16-bit vector instructions and parallelizes using a tree-algorithm to minimize dependencies. So already it's better than "pretty good" and it's still verbose. But neither of these compilers are able to optimize for code size while using AVX, so the ((code size)/64byte cacheline) * (~100ns loads) will still kill your performance on any data set that's smaller than a few kilobytes.
  - dragontamer 8y ago
    > The clang code loads the 8-bit words into 16-bit vector instructions and parallelizes using a tree-algorithm to minimize dependencies. You specified the "int sum", so that means "sum" needs to follow 32-bit overflow semantics. I don't think it is possible to do 8-bit > But neither of these compilers are able to optimize for code size while using AVX You can totally do that. You're specifying -O3, which is "speed over size". If you want size, you want -Os, and then use -ftree-vectorize to enable the auto-vectorization.

m0zg 8y ago

Staring is not enough, you need to benchmark. You might be surprised. The results are often counterintuitive, because it's hard for a human to account for how the program _actually_ gets executed, which instructions go in which order, port utilization, data dependencies, CPU bugs, effects of alignment (or, often, lack thereof on Intel CPUs), micro-op caching, etc etc. For all you know GCC deliberately avoided the use of AVX512 registers because AVX512 causes the CPU to throttle the clock. I've also found that ye olde -O2 flag recommendation which says -O3 is likely to be slower in practice because it produces bulkier machine code, nearly never holds up anymore. That said, I concede that autovectorization is not perfect. I can almost always beat it if I really want to, sometimes by quite a margin. But what you're proposing is unlikely to help matters, especially for smaller inputs that you're concerned about.

CoolGuySteve 8y ago
I have benchmarked similar code extensively, this is a toy version of a FIX checksum. It's not "counterintuitive", it's just broken, especially for infrequently executed code where the instruction fetch stalls will kill your latency. You are welcomed to load up the godbolt example and try and create better assembly using O2 or whatever else you can think of. You can build the code on your machine and benchmark it too. But at the end of the day, the compiler output sucks. It's clearly optimized for synthetic benchmarks that chug through large datasets using a tiny amount of code that can fit in L1 cache even after the compiler blows up its footprint. But my point is that these large datasets with tiny processing kernels (like a video codec for example) are much better suited to different kinds of processors.

dragontamer 8y ago

I don't believe Knights Landing supports an 8-bit ZMM add. Changing the code to 32-bit ints results in the "vpaddd zmm0, zmm0, ZMMWORD PTR [rdx]" that you'd expect from the auto-vectorizer. > You can see that the most recent GCC fails to use the AVX512 zmm registers even after being configured to do so (afaik) and also fails to use more than 4 registers In the 32-bit vpaddd code... "vpaddd zmm0, zmm0, ZMMWORD PTR [rdx]" becomes a new zmm register allocation in the register-renamer (due to a cut dependency). I doubt any code would be any faster. In any case, its not about "the number of registers used", proper analysis is about the depth of dependencies. I'd only be worried about small-register usage if the dependency chain is large (which doesn't seem to be the case). EDIT: I had some bad analysis. I've erased the bad paragraph. So it seems pretty good in my eyes.

CoolGuySteve 8y ago
The clang code loads the 8-bit words into 16-bit vector instructions and parallelizes using a tree-algorithm to minimize dependencies. So already it's better than "pretty good" and it's still verbose. But neither of these compilers are able to optimize for code size while using AVX, so the ((code size)/64byte cacheline) * (~100ns loads) will still kill your performance on any data set that's smaller than a few kilobytes.
- dragontamer 8y ago
  > The clang code loads the 8-bit words into 16-bit vector instructions and parallelizes using a tree-algorithm to minimize dependencies. You specified the "int sum", so that means "sum" needs to follow 32-bit overflow semantics. I don't think it is possible to do 8-bit > But neither of these compilers are able to optimize for code size while using AVX You can totally do that. You're specifying -O3, which is "speed over size". If you want size, you want -Os, and then use -ftree-vectorize to enable the auto-vectorization.