3 ms·

As a library author, I am very, very nervous about doing anything that might cause clock throttling. After all, you will not see a 2x gain from twice the SIMD w

As a library author, I am very, very nervous about doing anything that might cause clock throttling. After all, you will not see a 2x gain from twice the SIMD width (there are many other limiting factors, and of course, Skylake issues 2 x 512b operations max vs 3 x 256b ops). Even if you did get 2x, you would have to balance that against clock throttling in circumstances where you have no idea of the _relative_ importance of your library. Are you 95% of the cycles, or 10%?

lukego 7y ago
I'm always nervous about companies like Intel doing both hardware and software. If you do only hardware then you need to keep it simple or else software people won't use your stuff. If you are playing both sides then you can throw engineers at the problem and make it as complicated as you want. In this case they can just push complex AVX512 code into projects like glibc and bypass the whole "is this really worth the complexity?" discussion. One related example is Intel CPUs having good cache-DMA integration but this not working across NUMA nodes. Ideally Intel would improve this feature to be NUMA-aware and everybody would be happy. However, it seems like they decided it would be cheaper to send forth an army of software engineers to put NUMA-scheduling kludges into a few key projects like Kubernetes and then call it a day. /rant!
- tntn 7y ago
  > good cache-DMA integration but this not working across NUMA nodes. Ideally Intel would improve this feature to be NUMA-aware How do you propose they do this? You can't magically move the physical devices to the other socket, and transferring the data between sockets is what those "NUMA-scheduling kludges" are trying to avoid. The only solution is to have software put the data close to the device.
  - lukego 7y ago
    My take is that they should use the memory subsystem to route the DMA to the appropriate L3 cache. I don't think the main issue is conserving QPI bandwidth: it's minimizing latency when a core loads the data into L2. That latency often causes ~30% performance drag on e.g. packet processing applications.
gameswithgo 7y ago
sometimes you get more than a 2x gain when going to a wider simd instruction set, because more instructions are available. This is especially true with AVX-512 and its masks.

lukego 7y ago

I'm always nervous about companies like Intel doing both hardware and software. If you do only hardware then you need to keep it simple or else software people won't use your stuff. If you are playing both sides then you can throw engineers at the problem and make it as complicated as you want. In this case they can just push complex AVX512 code into projects like glibc and bypass the whole "is this really worth the complexity?" discussion. One related example is Intel CPUs having good cache-DMA integration but this not working across NUMA nodes. Ideally Intel would improve this feature to be NUMA-aware and everybody would be happy. However, it seems like they decided it would be cheaper to send forth an army of software engineers to put NUMA-scheduling kludges into a few key projects like Kubernetes and then call it a day. /rant!

tntn 7y ago
> good cache-DMA integration but this not working across NUMA nodes. Ideally Intel would improve this feature to be NUMA-aware How do you propose they do this? You can't magically move the physical devices to the other socket, and transferring the data between sockets is what those "NUMA-scheduling kludges" are trying to avoid. The only solution is to have software put the data close to the device.
- lukego 7y ago
  My take is that they should use the memory subsystem to route the DMA to the appropriate L3 cache. I don't think the main issue is conserving QPI bandwidth: it's minimizing latency when a core loads the data into L2. That latency often causes ~30% performance drag on e.g. packet processing applications.

gameswithgo 7y ago

sometimes you get more than a 2x gain when going to a wider simd instruction set, because more instructions are available. This is especially true with AVX-512 and its masks.