4 ms·

I often wonder how much performance is being left behind from these extensions not being used explicitly (are compilers using these extensions automatically?).

I often wonder how much performance is being left behind from these extensions not being used explicitly (are compilers using these extensions automatically?). SSE, AVX.. Making it simplier would seem to be a huge win as people start to use this clearly improves performance significantly. Its seems most new CPUs support a fairly large subset of the older ones. I haven't dont low lever work of a while but remember doing some analysis of a signal processing code going from PA-RISC to Intel. The math compiler libraries for PA-RISC made the initial code transition so much slower on Intel. Using the Intel compiler and tweaking things started making things work so much better.

colejohnson66 5y ago
The big problem with Intel’s compiler is that their dispatch function requires the CPU identify itself as “GenuineIntel” (for the vectorization advantage), which AMD processors don’t do. https://www.agner.org/optimize/blog/read.php?i=49 https://www.agner.org/optimize/blog/read.php?i=49
- deleted 5y ago
  [deleted]
- acomjean 5y ago
  Interesting. and kinda sad. and based on the comments an ongoing problem. One hopes that ARM's ascendance would make "team x86-64" work in cooperative competition for better performance through compilers (it they can't through silicon).
  - arthur2e5 5y ago
    AMD and recently Intel both moving to LLVM is a good step in that direction IMO. LLVM also already comes with a dispatcher, so hopefully they aren’t going to… do it again. Personally I’m more excited about being able to port __attribute__ rich code more easily. (IIRC Intel has tuned down the Cripple AMD thing in MKL around late 2020 by providing a specialized Zen code path. It was slower than what you get with the detection backdoor, but only slightly.)
arthur2e5 5y ago
Compilers are using them,^1 but aren’t necessarily using them enough. They (generally) can’t change alignments of a struct willy-nilly, nor can they just, say, split a big FP reduction into many parallel ones without violating 754. In most loop-related cases however, I would say a nudge in the form of #pragma omp simd goes a long way.^2 And then there’s the genius stuff like SIMD UTF8 decode and whatnot. Compilers don’t magically figure those out. Heck, they sometimes have trouble with shuffles, which is why the author mentions that the new features will help. ^1 Oh, not GCC on -O2 IIRC. There were some talks about turning autovec on that level on like clang does, but I’m not sure it went anywhere… I also think MSVC doesn’t do that. ^2 This pragma also turns on autovec even if the compiler is not told to do that with the flags. Although my original point was about the extra information you can provide with it.
beached_whale 5y ago
I was recently wondering how much CPU time would be regained if we got an instruction to do x * 10 ^ y where x is an int64/y an int (or something like that) and do so without error. This is a really really common operation and needs a bigint library to do properly now for many cases. It’s also slow, I think aggregate it’s around 500MB/s to 1GB/s, but the slow path is < 100MB/s. But pretty much every JSON/XML/Number.from_string is using an operation like this, to the scale is quite large.
- stephencanon 5y ago
  Converting a number from string can be done without any bignum operations at all, this has been well-understood for a while now. This is mostly just widely-used libraries having not caught up with the state of the art. There is some opportunity for ISA support to speed it up, but multiplication by powers of ten is not the bottleneck, that’s just a table lookup and full-width product in high-performance implementations (i.e. a load and a mul instruction).
  - beached_whale 5y ago
    Got links? Even Daniel Lemire's Fast Double parser will dump to strtod when the numbers are not handleable because of this. You cannot represent 10^x, or even all 5^x beyond a certain point in IEEE754 double's but need to do the full y * 10^x operation without loss. Doing it lossy is easy(0 to 1 or 2 ulp off optimal)
    - stephencanon 5y ago
      You don’t need to do it without loss, you need to do it with loss smaller than the closest that the digit sequence could be to an exact halfway point between two representable double-precision values. This means you may need arbitrary-precision for arbitrary-precision decimal strings, but in practice these libraries are “always” converting strings that came from formatting exact doubles (eg round-tripping through JSON), and so have bounded precision. Thus you can tightly bound the precision required. This precisely mirrors how formatting fp numbers used to require bignum arithmetic, but all the recent algorithms (Ryu, Dragonbox, SwiftDtoa) do it with fixed int128 arithmetic and deliver always correct results. We’ll see analogous algorithms for the other direction adopted in the next few years—the only reason this direction came later is that the other was a bigger performance problem originally.
jeffbee 5y ago
It's all application-specific. It's not even true that compiling with -march=${cpu} gives you the best performance on that CPU. As an example it was once discovered that for a certain large program -march=haswell was counterproductive on Haswell, and choosing a subset of those features was faster in practice. The same turned out to be true for Skylake. And BMI2 did not work right on AMD processors until very recently, so the fact that the PDEP and PEXT existed was not helpful. Your program would run, but the instructions had high and data-dependent latency. In short, you need to measure the combination of compiler options that get you the best performance on your real platform. Most people can probably pick up a quick 10-20% win by recompiling their MySQL or whatever for arch=sandybridge, instead of k8-generic, but beyond that it gets trickier.
- dataflow 5y ago
  > It's not even true that compiling with -march=${cpu} gives you the best performance on that CPU Are you thinking of -mtune? I'd never heard of people using -march for performance, I thought it was just specifying the ISA.
  - jeffbee 5y ago
    -march tells the compiler which instruction set extensions it is allowed to use. For e.g. the difference between arch=haswell and arch=sandybridge is AVX2, MOVBE, FMA, BMI, and BMI2. If you build with haswell, or any given CPU, the compiler might emit instructions that's won't run on older hardware. That's why most Linux distributions for PC are compiled for k8-generic, an x86-64 platform with few extensions (MMX, SSE, and SSE2, if I recall correctly). In the particular case as I recall it was AVX2 that was counterproductive on haswell. Disabling it made the program faster, even though AVX2 was supposed to be a headline feature of that generation.
NohatCoder 5y ago
In general, almost all of it. If you haven't laid out your data to make it accessible to SIMD instructions, at best the program will get a modest speedup by using most of the instructions rearranging data to be SIMD-compatible. By far the simplest way of making the correct data-layout is to use the intrinsics manually, so that you discover any issues that arise. A step further is to change the program logic to better take advantage of SIMD-instructions, for instance Fabian Giesen has written a great deal on making compression code that splices the jobs in creative ways in order to utilize SIMD-instructions.

colejohnson66 5y ago

The big problem with Intel’s compiler is that their dispatch function requires the CPU identify itself as “GenuineIntel” (for the vectorization advantage), which AMD processors don’t do. https://www.agner.org/optimize/blog/read.php?i=49 https://www.agner.org/optimize/blog/read.php?i=49

deleted 5y ago
[deleted]
acomjean 5y ago
Interesting. and kinda sad. and based on the comments an ongoing problem. One hopes that ARM's ascendance would make "team x86-64" work in cooperative competition for better performance through compilers (it they can't through silicon).
- arthur2e5 5y ago
  AMD and recently Intel both moving to LLVM is a good step in that direction IMO. LLVM also already comes with a dispatcher, so hopefully they aren’t going to… do it again. Personally I’m more excited about being able to port __attribute__ rich code more easily. (IIRC Intel has tuned down the Cripple AMD thing in MKL around late 2020 by providing a specialized Zen code path. It was slower than what you get with the detection backdoor, but only slightly.)

arthur2e5 5y ago

Compilers are using them,^1 but aren’t necessarily using them enough. They (generally) can’t change alignments of a struct willy-nilly, nor can they just, say, split a big FP reduction into many parallel ones without violating 754. In most loop-related cases however, I would say a nudge in the form of #pragma omp simd goes a long way.^2 And then there’s the genius stuff like SIMD UTF8 decode and whatnot. Compilers don’t magically figure those out. Heck, they sometimes have trouble with shuffles, which is why the author mentions that the new features will help. ^1 Oh, not GCC on -O2 IIRC. There were some talks about turning autovec on that level on like clang does, but I’m not sure it went anywhere… I also think MSVC doesn’t do that. ^2 This pragma also turns on autovec even if the compiler is not told to do that with the flags. Although my original point was about the extra information you can provide with it.

beached_whale 5y ago

I was recently wondering how much CPU time would be regained if we got an instruction to do x * 10 ^ y where x is an int64/y an int (or something like that) and do so without error. This is a really really common operation and needs a bigint library to do properly now for many cases. It’s also slow, I think aggregate it’s around 500MB/s to 1GB/s, but the slow path is < 100MB/s. But pretty much every JSON/XML/Number.from_string is using an operation like this, to the scale is quite large.

stephencanon 5y ago
Converting a number from string can be done without any bignum operations at all, this has been well-understood for a while now. This is mostly just widely-used libraries having not caught up with the state of the art. There is some opportunity for ISA support to speed it up, but multiplication by powers of ten is not the bottleneck, that’s just a table lookup and full-width product in high-performance implementations (i.e. a load and a mul instruction).
- beached_whale 5y ago
  Got links? Even Daniel Lemire's Fast Double parser will dump to strtod when the numbers are not handleable because of this. You cannot represent 10^x, or even all 5^x beyond a certain point in IEEE754 double's but need to do the full y * 10^x operation without loss. Doing it lossy is easy(0 to 1 or 2 ulp off optimal)
  - stephencanon 5y ago
    You don’t need to do it without loss, you need to do it with loss smaller than the closest that the digit sequence could be to an exact halfway point between two representable double-precision values. This means you may need arbitrary-precision for arbitrary-precision decimal strings, but in practice these libraries are “always” converting strings that came from formatting exact doubles (eg round-tripping through JSON), and so have bounded precision. Thus you can tightly bound the precision required. This precisely mirrors how formatting fp numbers used to require bignum arithmetic, but all the recent algorithms (Ryu, Dragonbox, SwiftDtoa) do it with fixed int128 arithmetic and deliver always correct results. We’ll see analogous algorithms for the other direction adopted in the next few years—the only reason this direction came later is that the other was a bigger performance problem originally.

jeffbee 5y ago

It's all application-specific. It's not even true that compiling with -march=${cpu} gives you the best performance on that CPU. As an example it was once discovered that for a certain large program -march=haswell was counterproductive on Haswell, and choosing a subset of those features was faster in practice. The same turned out to be true for Skylake. And BMI2 did not work right on AMD processors until very recently, so the fact that the PDEP and PEXT existed was not helpful. Your program would run, but the instructions had high and data-dependent latency. In short, you need to measure the combination of compiler options that get you the best performance on your real platform. Most people can probably pick up a quick 10-20% win by recompiling their MySQL or whatever for arch=sandybridge, instead of k8-generic, but beyond that it gets trickier.

dataflow 5y ago
> It's not even true that compiling with -march=${cpu} gives you the best performance on that CPU Are you thinking of -mtune? I'd never heard of people using -march for performance, I thought it was just specifying the ISA.
- jeffbee 5y ago
  -march tells the compiler which instruction set extensions it is allowed to use. For e.g. the difference between arch=haswell and arch=sandybridge is AVX2, MOVBE, FMA, BMI, and BMI2. If you build with haswell, or any given CPU, the compiler might emit instructions that's won't run on older hardware. That's why most Linux distributions for PC are compiled for k8-generic, an x86-64 platform with few extensions (MMX, SSE, and SSE2, if I recall correctly). In the particular case as I recall it was AVX2 that was counterproductive on haswell. Disabling it made the program faster, even though AVX2 was supposed to be a headline feature of that generation.

NohatCoder 5y ago

In general, almost all of it. If you haven't laid out your data to make it accessible to SIMD instructions, at best the program will get a modest speedup by using most of the instructions rearranging data to be SIMD-compatible. By far the simplest way of making the correct data-layout is to use the intrinsics manually, so that you discover any issues that arise. A step further is to change the program logic to better take advantage of SIMD-instructions, for instance Fabian Giesen has written a great deal on making compression code that splices the jobs in creative ways in order to utilize SIMD-instructions.