4 ms·

Function multi-versioning in GCC 6 (2016)

29 points by pkkm 3y ago

jeffbee 3y ago
Hopefully we are looking back at GCC ifunc as a mistake / warning to future civilizations. Indirectly dispatching functions via the PLT only makes sense for operations where the µarch specialization is sure to be profitable, like bulk cryptography or string ops statically determined to have large inputs. But dispatch to a specialized AVX memcpy for inputs of unknown length in not profitable. The dispatch mechanism costs more than the specialization saves. Unfortunately ifunc usually looks great on microbenchmarks and the drawbacks don't become apparent until observed in a full-scale program.
- dundarious 3y ago
  I'm an "auto-vectorization skeptic" in many ways, but it's also the case that there will be other parts of your program that will benefit from being able to generate (for example) AVX2 instructions beyond a few specialized functions. If there is to be a CPU-specific dispatch mechanism, it should be more like a small binary/shell script that exec-s an entirely CPU-specific binary, dependent on runtime detection (cpuid, etc.). Dependent on your distribution mechanism, you might only download the appropriate binary. But if you do this, please remember to keep your binary sizes reasonable.
  - gumby 3y ago
    I hope your skepticism is appropriately language-specific. For example auto-vectorization of FORTRAN code has a long, successful history (the Cray 1, and X/Y-MP were basically FORTRAN machines) due to its memory model and vector affordances. With C, C++, Rust and other such languages of course it's very difficult.
    - dundarious 3y ago
      Fortran was safely excluded.
- saagarjha 3y ago
  This doesn’t make sense to me. memcpy is typically called through the PLT anyways; all the ifunc does is change what it points at. Even if there was some extra cost here one indirect call is not that expensive?
  - jeffbee 3y ago
    mem* do not need to be called via ifunc. That is a toolchain decision. See e.g. https://github.com/nadavrot/memset_benchmark https://github.com/nadavrot/memset_benchmark for recent data about the cost of PLT indirection for small copies. ETA: Also this excellent paper that is all about optimizing mem* notes that PLT cost adds up. https://dl.acm.org/doi/pdf/10.1145/3459898.3463904 https://dl.acm.org/doi/pdf/10.1145/3459898.3463904
    - saagarjha 3y ago
      I’ve read this paper actually! It’s pretty good, but needs to be taken in context. It’s clear that going through the PLT has a cost, which becomes more pronounced as the actual work done in the function goes down because it’s a fixed overhead. If you’re doing a store or two because the size is really small it makes perfect sense that anything that needs to be done beyond that is going to be measurable. This is true even for a direct function call, too, though the alternatives to that are generally not desirable for almost all workloads. For some clients it makes sense to statically link an implementation of these functions into your program. If you’re Google and your machines run one binary all day where you know the exact hardware they’re using, go wild with the assumptions that get baked into it. For everyone else, though, it is generally not the case that you want to reimplement libc, nor will you distribute separate binaries for each microarchitecure. In this case, you’ll pay the cost of linking against the system libraries anyways, and ifuncs give you a way to fold the cost of selecting hardware-specific optimizations into this process. This is why both glibc and libplatform do it. As far as I’m aware Microsoft chooses to actually put the check at the point where the optimized instructions would get used, but maybe that works better with how they do linking on that platform; I don’t know enough about it to have any insight to share.
10000truths 3y ago
Too bad it's tied to glibc. If you want to link with musl, you need to define the dynamic dispatch logic manually.

29 points by pkkm 3y ago

Hopefully we are looking back at GCC ifunc as a mistake / warning to future civilizations. Indirectly dispatching functions via the PLT only makes sense for operations where the µarch specialization is sure to be profitable, like bulk cryptography or string ops statically determined to have large inputs. But dispatch to a specialized AVX memcpy for inputs of unknown length in not profitable. The dispatch mechanism costs more than the specialization saves. Unfortunately ifunc usually looks great on microbenchmarks and the drawbacks don't become apparent until observed in a full-scale program.

dundarious 3y ago
I'm an "auto-vectorization skeptic" in many ways, but it's also the case that there will be other parts of your program that will benefit from being able to generate (for example) AVX2 instructions beyond a few specialized functions. If there is to be a CPU-specific dispatch mechanism, it should be more like a small binary/shell script that exec-s an entirely CPU-specific binary, dependent on runtime detection (cpuid, etc.). Dependent on your distribution mechanism, you might only download the appropriate binary. But if you do this, please remember to keep your binary sizes reasonable.
- gumby 3y ago
  I hope your skepticism is appropriately language-specific. For example auto-vectorization of FORTRAN code has a long, successful history (the Cray 1, and X/Y-MP were basically FORTRAN machines) due to its memory model and vector affordances. With C, C++, Rust and other such languages of course it's very difficult.
  - dundarious 3y ago
    Fortran was safely excluded.
saagarjha 3y ago
This doesn’t make sense to me. memcpy is typically called through the PLT anyways; all the ifunc does is change what it points at. Even if there was some extra cost here one indirect call is not that expensive?
- jeffbee 3y ago
  mem* do not need to be called via ifunc. That is a toolchain decision. See e.g. https://github.com/nadavrot/memset_benchmark https://github.com/nadavrot/memset_benchmark for recent data about the cost of PLT indirection for small copies. ETA: Also this excellent paper that is all about optimizing mem* notes that PLT cost adds up. https://dl.acm.org/doi/pdf/10.1145/3459898.3463904 https://dl.acm.org/doi/pdf/10.1145/3459898.3463904
  - saagarjha 3y ago
    I’ve read this paper actually! It’s pretty good, but needs to be taken in context. It’s clear that going through the PLT has a cost, which becomes more pronounced as the actual work done in the function goes down because it’s a fixed overhead. If you’re doing a store or two because the size is really small it makes perfect sense that anything that needs to be done beyond that is going to be measurable. This is true even for a direct function call, too, though the alternatives to that are generally not desirable for almost all workloads. For some clients it makes sense to statically link an implementation of these functions into your program. If you’re Google and your machines run one binary all day where you know the exact hardware they’re using, go wild with the assumptions that get baked into it. For everyone else, though, it is generally not the case that you want to reimplement libc, nor will you distribute separate binaries for each microarchitecure. In this case, you’ll pay the cost of linking against the system libraries anyways, and ifuncs give you a way to fold the cost of selecting hardware-specific optimizations into this process. This is why both glibc and libplatform do it. As far as I’m aware Microsoft chooses to actually put the check at the point where the optimized instructions would get used, but maybe that works better with how they do linking on that platform; I don’t know enough about it to have any insight to share.

10000truths 3y ago

Too bad it's tied to glibc. If you want to link with musl, you need to define the dynamic dispatch logic manually.