8 ms·

Thanks! While you are here... :-) What do you think about the tendency of glibc to use the latest available instruction set extensions for routines like memcp

by lukego 7y ago

Thanks! While you are here... :-) What do you think about the tendency of glibc to use the latest available instruction set extensions for routines like memcpy? Should we be worried that when we call memcpy() in a program this will effectively mix some wide-datapath instructions into the CPU's instruction stream with potentially hilarious consequences like clock throttling? What's all that about?

glangdale 7y ago
As a library author, I am very, very nervous about doing anything that might cause clock throttling. After all, you will not see a 2x gain from twice the SIMD width (there are many other limiting factors, and of course, Skylake issues 2 x 512b operations max vs 3 x 256b ops). Even if you did get 2x, you would have to balance that against clock throttling in circumstances where you have no idea of the _relative_ importance of your library. Are you 95% of the cycles, or 10%?
- lukego 7y ago
  I'm always nervous about companies like Intel doing both hardware and software. If you do only hardware then you need to keep it simple or else software people won't use your stuff. If you are playing both sides then you can throw engineers at the problem and make it as complicated as you want. In this case they can just push complex AVX512 code into projects like glibc and bypass the whole "is this really worth the complexity?" discussion. One related example is Intel CPUs having good cache-DMA integration but this not working across NUMA nodes. Ideally Intel would improve this feature to be NUMA-aware and everybody would be happy. However, it seems like they decided it would be cheaper to send forth an army of software engineers to put NUMA-scheduling kludges into a few key projects like Kubernetes and then call it a day. /rant!
  - tntn 7y ago
    > good cache-DMA integration but this not working across NUMA nodes. Ideally Intel would improve this feature to be NUMA-aware How do you propose they do this? You can't magically move the physical devices to the other socket, and transferring the data between sockets is what those "NUMA-scheduling kludges" are trying to avoid. The only solution is to have software put the data close to the device.
    - lukego 7y ago
      My take is that they should use the memory subsystem to route the DMA to the appropriate L3 cache. I don't think the main issue is conserving QPI bandwidth: it's minimizing latency when a core loads the data into L2. That latency often causes ~30% performance drag on e.g. packet processing applications.
- gameswithgo 7y ago
  sometimes you get more than a 2x gain when going to a wider simd instruction set, because more instructions are available. This is especially true with AVX-512 and its masks.
jabl 7y ago
IIRC glibc reverted those avx-512 string functions, precisely because it turned out that in most real programs outside microbenchmarks, the clock speed hit negated the benefit from wider instructions. But IIRC, back when AVX was introduced, there was some issue with that as well, and similar with AVX2. But now those seem to have been mostly addressed. I guess in some years the same will happen with AVX-512 too?
- lukego 7y ago
  Is there definitive information on these issues? The pattern for me is that I'm excited about a new instruction set when it is announced, then disappointed when it is shipped in silicon, then I have a hard time knowing when it is truly resolved and worth taking a second look at. My recollection with AVX2 is that glibc shipped those routines since before silicon was generally available, and then kept them enabled even when the clock-throttling issues became apparent. (The water was muddied a bit in one glibc release that shipped a bug in selecting which code to use for the running uarch, but that's a separate issue.)
  - mbell 7y ago
    Not specifically glibc but some data on the frequency scaling impact: https://blog.cloudflare.com/on-the-dangers-of-intels-frequency-scaling/ https://blog.cloudflare.com/on-the-dangers-of-intels-frequen...

by lukego 7y ago

glangdale 7y ago

As a library author, I am very, very nervous about doing anything that might cause clock throttling. After all, you will not see a 2x gain from twice the SIMD width (there are many other limiting factors, and of course, Skylake issues 2 x 512b operations max vs 3 x 256b ops). Even if you did get 2x, you would have to balance that against clock throttling in circumstances where you have no idea of the _relative_ importance of your library. Are you 95% of the cycles, or 10%?

lukego 7y ago
I'm always nervous about companies like Intel doing both hardware and software. If you do only hardware then you need to keep it simple or else software people won't use your stuff. If you are playing both sides then you can throw engineers at the problem and make it as complicated as you want. In this case they can just push complex AVX512 code into projects like glibc and bypass the whole "is this really worth the complexity?" discussion. One related example is Intel CPUs having good cache-DMA integration but this not working across NUMA nodes. Ideally Intel would improve this feature to be NUMA-aware and everybody would be happy. However, it seems like they decided it would be cheaper to send forth an army of software engineers to put NUMA-scheduling kludges into a few key projects like Kubernetes and then call it a day. /rant!
- tntn 7y ago
  > good cache-DMA integration but this not working across NUMA nodes. Ideally Intel would improve this feature to be NUMA-aware How do you propose they do this? You can't magically move the physical devices to the other socket, and transferring the data between sockets is what those "NUMA-scheduling kludges" are trying to avoid. The only solution is to have software put the data close to the device.
  - lukego 7y ago
    My take is that they should use the memory subsystem to route the DMA to the appropriate L3 cache. I don't think the main issue is conserving QPI bandwidth: it's minimizing latency when a core loads the data into L2. That latency often causes ~30% performance drag on e.g. packet processing applications.
gameswithgo 7y ago
sometimes you get more than a 2x gain when going to a wider simd instruction set, because more instructions are available. This is especially true with AVX-512 and its masks.

jabl 7y ago

IIRC glibc reverted those avx-512 string functions, precisely because it turned out that in most real programs outside microbenchmarks, the clock speed hit negated the benefit from wider instructions. But IIRC, back when AVX was introduced, there was some issue with that as well, and similar with AVX2. But now those seem to have been mostly addressed. I guess in some years the same will happen with AVX-512 too?

lukego 7y ago
Is there definitive information on these issues? The pattern for me is that I'm excited about a new instruction set when it is announced, then disappointed when it is shipped in silicon, then I have a hard time knowing when it is truly resolved and worth taking a second look at. My recollection with AVX2 is that glibc shipped those routines since before silicon was generally available, and then kept them enabled even when the clock-throttling issues became apparent. (The water was muddied a bit in one glibc release that shipped a bug in selecting which code to use for the running uarch, but that's a separate issue.)
- mbell 7y ago
  Not specifically glibc but some data on the frequency scaling impact: https://blog.cloudflare.com/on-the-dangers-of-intels-frequency-scaling/ https://blog.cloudflare.com/on-the-dangers-of-intels-frequen...