5 ms·

Are there good libraries for doing runtime feature detection? Eg, include three versions of hot function X in the binary, and have it seamlessly insert the corr

Are there good libraries for doing runtime feature detection? Eg, include three versions of hot function X in the binary, and have it seamlessly insert the correct function pointer at startup? Or have the function contain multiple bodies and just JMP to the correct block of code? I know you can do this yourself, but last time I looked it was a heavily manual process— you had to basically define a plugin interface and dynamically load your selected implementation from a separate shared object. What are the barriers to having compilers able to be hinted into transparently generating multiple versions of key functions?

colejohnson66 4y ago
Check out Agner Fog's vectorclass library: https://github.com/vectorclass/version2 https://github.com/vectorclass/version2
bremac 4y ago
I'm unsure about library support, but gcc and clang support function multi-versioning (FMV), which resolves the function based on CPUID the first time the function is called. This LWN article has some additional information: https://lwn.net/Articles/691932/ https://lwn.net/Articles/691932/
- mikepurvis 4y ago
  TIL! I guess it makes sense that popular numeric libraries like BLAS, Eigen, and so-on would take advantage of this, but I wonder how widely used it is overall.
  - bee_rider 4y ago
    For BLAS, it has enough legacy that the vendors just make their own libraries, so it would probably be best to use Intel MKL (which has it's own internal dispatching, but I don't think it uses FMV) if you are targeting Intel, and maybe AMD's branch of BLIS if you are targeting AMD. I believe Eigen comes with it's own version of BLAS built in, but it can be made to link to MKL at least, probably BLIS if you are really stubborn.
loeg 4y ago
GCC has offered Function Multiversioning for about a decade now (GCC ~4.8 or 4.9). GCC 6's resolver apparently uses CPUID to resolve the ifunc once at program start: https://lwn.net/Articles/691932/ https://lwn.net/Articles/691932/ . Clang added it in 7.0.0: https://releases.llvm.org/7.0.0/tools/clang/docs/AttributeReference.html#target-gnu-target https://releases.llvm.org/7.0.0/tools/clang/docs/AttributeRe... A nice presentation on it: https://llvm.org/devmtg/2014-10/Slides/Christopher-Function%20Multiversioning%20Talk.pdf https://llvm.org/devmtg/2014-10/Slides/Christopher-Function%...
- indygreg2 4y ago
  While this IFUNC feature does exist and it is useful, when I performed binary analysis on every package in Ubuntu in January, I found that only ~11 distinct packages have IFUNCs. It certainly looks like this ELF feature is not really used much [in open source] outside of GNU toolchain-level software! https://gregoryszorc.com/blog/2022/01/09/bulk-analyze-linux-packages-with-linux-package-analyzer/ https://gregoryszorc.com/blog/2022/01/09/bulk-analyze-linux-...
  - TkTech 4y ago
    I've wanted to use them many times in the past, but the limited support on other compilers (looking at you MSVC) always made it a non-starter. If I have to support some other method of feature detection anyways, there's no point.
  - jeffbee 4y ago
    The use of ifunc in libc, openssl, and zlib covers just about everything people want from it. You want that optimized memcpy, you only need it in one place.
  - arthur2e5 4y ago
    The way ifunc (well, actually language-level FMV) works in GCC and clang is that the input source code, not the command-line switches, specifies what ISA extensions to build for on a candidate function. This naturally means that packagers and other vendors are not using it as much as you hope: they would need to have a separate patch to add these attributes for each architecture and that’s just not maintainable. Even Intel’s Clear Linux does not bother to patch individual libraries. They just use the glibc library multi-versioning feature to load from different directories depending on the cpuid. In my opinion, GCC and Clang could make the whole thing more ergonomic. Ideally you declare a function as "interesting for multi-versioning" in the source using an attribute, then in the command line define what -march's to actually clone for. Kinda like ICC’s /Qax. (On second thought preprocessor defs are sufficient, duh.)
Scene_Cast2 4y ago
For vectorizable workloads, there's Halide. It can be set up in multiple ways, one of which is a JIT that auto-detects CPU capabilities and cache sizes (to better optimize scheduling).

colejohnson66 4y ago

Check out Agner Fog's vectorclass library: https://github.com/vectorclass/version2 https://github.com/vectorclass/version2

bremac 4y ago

I'm unsure about library support, but gcc and clang support function multi-versioning (FMV), which resolves the function based on CPUID the first time the function is called. This LWN article has some additional information: https://lwn.net/Articles/691932/ https://lwn.net/Articles/691932/

mikepurvis 4y ago
TIL! I guess it makes sense that popular numeric libraries like BLAS, Eigen, and so-on would take advantage of this, but I wonder how widely used it is overall.
- bee_rider 4y ago
  For BLAS, it has enough legacy that the vendors just make their own libraries, so it would probably be best to use Intel MKL (which has it's own internal dispatching, but I don't think it uses FMV) if you are targeting Intel, and maybe AMD's branch of BLIS if you are targeting AMD. I believe Eigen comes with it's own version of BLAS built in, but it can be made to link to MKL at least, probably BLIS if you are really stubborn.

loeg 4y ago

GCC has offered Function Multiversioning for about a decade now (GCC ~4.8 or 4.9). GCC 6's resolver apparently uses CPUID to resolve the ifunc once at program start: https://lwn.net/Articles/691932/ https://lwn.net/Articles/691932/ . Clang added it in 7.0.0: https://releases.llvm.org/7.0.0/tools/clang/docs/AttributeReference.html#target-gnu-target https://releases.llvm.org/7.0.0/tools/clang/docs/AttributeRe... A nice presentation on it: https://llvm.org/devmtg/2014-10/Slides/Christopher-Function%20Multiversioning%20Talk.pdf https://llvm.org/devmtg/2014-10/Slides/Christopher-Function%...

indygreg2 4y ago
While this IFUNC feature does exist and it is useful, when I performed binary analysis on every package in Ubuntu in January, I found that only ~11 distinct packages have IFUNCs. It certainly looks like this ELF feature is not really used much [in open source] outside of GNU toolchain-level software! https://gregoryszorc.com/blog/2022/01/09/bulk-analyze-linux-packages-with-linux-package-analyzer/ https://gregoryszorc.com/blog/2022/01/09/bulk-analyze-linux-...
- TkTech 4y ago
  I've wanted to use them many times in the past, but the limited support on other compilers (looking at you MSVC) always made it a non-starter. If I have to support some other method of feature detection anyways, there's no point.
- jeffbee 4y ago
  The use of ifunc in libc, openssl, and zlib covers just about everything people want from it. You want that optimized memcpy, you only need it in one place.
- arthur2e5 4y ago
  The way ifunc (well, actually language-level FMV) works in GCC and clang is that the input source code, not the command-line switches, specifies what ISA extensions to build for on a candidate function. This naturally means that packagers and other vendors are not using it as much as you hope: they would need to have a separate patch to add these attributes for each architecture and that’s just not maintainable. Even Intel’s Clear Linux does not bother to patch individual libraries. They just use the glibc library multi-versioning feature to load from different directories depending on the cpuid. In my opinion, GCC and Clang could make the whole thing more ergonomic. Ideally you declare a function as "interesting for multi-versioning" in the source using an attribute, then in the command line define what -march's to actually clone for. Kinda like ICC’s /Qax. (On second thought preprocessor defs are sufficient, duh.)

Scene_Cast2 4y ago

For vectorizable workloads, there's Halide. It can be set up in multiple ways, one of which is a JIT that auto-detects CPU capabilities and cache sizes (to better optimize scheduling).