3 ms·

Be very careful about this in a shared environment. AVX512 slows down the CPU cores because of thermal, voltage throttling. The instructions "take more work". I

by sargun 9y ago

Be very careful about this in a shared environment. AVX512 slows down the CPU cores because of thermal, voltage throttling. The instructions "take more work". Intel CPUs take 1 MILLIsecond to return to normal speed. If you're doing any kind of rapid context switching, or multiple workloads, depending on how your scheduler is setup, the OTHER workloads will show up as using more percentage of CPU time per work item. It's non-intuitive, and difficult to debug.

stu2010 9y ago
Now that major cloud vendors are selling VMs with guaranteed AVX512 support, how are they going to deal with the "noisy neighbor" problem?
- desdiv 9y ago
  They don't? AFAIK no cloud vendor guarantees consistent performance on their multi-tenancy offerings (at least none backed by SLAs). Vendors sell both dedicated tenancy, with consistent performance, at a higher price and shared tenancy, with variable performance, at a lower price and expect customers who choose the latter to deal with the variance on their own. Large companies run a benchmark after each instance startup to check how noisy that instance's neighbors are. If it's bad then they simply terminate the instance and start another one[0]. [0] https://www.reddit.com/r/aws/comments/547xbx/netflix_found_5x_performance_variation_between/ https://www.reddit.com/r/aws/comments/547xbx/netflix_found_5...
- microcolonel 9y ago
  Giant heatsinks, and more sockets! More likely though, they'll continue not to care. Performance is already hideously variable, it'll just get a bit worse.
- sargun 9y ago
  For one, this is done on a core-by-core level. I believe Intel has the capability to monitor for this behaviour and do some dynamic throttling in certain models, which cloud vendors may have access to. I think they introduced some of this in Broadwell-E, where you can set the AVX offset per core, and the on-die PCU will throttle that core below the AVX base frequency, allowing the rest of the cores to remain at speed. These are typically controlled by BIOS, or MSRs. We're currently trying to figure out how to do deal with this. Some ideas come from Google's CPI2 paper, and trying to dynamically schedule workloads with diversity if we think they interfere. Other thoughts have been simpler, like core pinning (knapsacking for latency, or throughput). this is hard. Disclaimer: These views represent my own, and not my employer's, or their vendor's views
zurn 9y ago
This doesn't use AVX512 though, and AVX512 is not supported in any mainstream JVM or CPU. It's only available in Xeon Phi, a HPC accelerator card with many slow cores.
- sargun 9y ago
  AVX and AVX2 are also included.
zbjornson 9y ago
CPUs throttle down for AVX and AVX2 workloads as well. Hopefully your vectorization is sufficient to make up for the throttling.

by sargun 9y ago

stu2010 9y ago

Now that major cloud vendors are selling VMs with guaranteed AVX512 support, how are they going to deal with the "noisy neighbor" problem?

desdiv 9y ago
They don't? AFAIK no cloud vendor guarantees consistent performance on their multi-tenancy offerings (at least none backed by SLAs). Vendors sell both dedicated tenancy, with consistent performance, at a higher price and shared tenancy, with variable performance, at a lower price and expect customers who choose the latter to deal with the variance on their own. Large companies run a benchmark after each instance startup to check how noisy that instance's neighbors are. If it's bad then they simply terminate the instance and start another one[0]. [0] https://www.reddit.com/r/aws/comments/547xbx/netflix_found_5x_performance_variation_between/ https://www.reddit.com/r/aws/comments/547xbx/netflix_found_5...
microcolonel 9y ago
Giant heatsinks, and more sockets! More likely though, they'll continue not to care. Performance is already hideously variable, it'll just get a bit worse.
sargun 9y ago
For one, this is done on a core-by-core level. I believe Intel has the capability to monitor for this behaviour and do some dynamic throttling in certain models, which cloud vendors may have access to. I think they introduced some of this in Broadwell-E, where you can set the AVX offset per core, and the on-die PCU will throttle that core below the AVX base frequency, allowing the rest of the cores to remain at speed. These are typically controlled by BIOS, or MSRs. We're currently trying to figure out how to do deal with this. Some ideas come from Google's CPI2 paper, and trying to dynamically schedule workloads with diversity if we think they interfere. Other thoughts have been simpler, like core pinning (knapsacking for latency, or throughput). this is hard. Disclaimer: These views represent my own, and not my employer's, or their vendor's views

zurn 9y ago

This doesn't use AVX512 though, and AVX512 is not supported in any mainstream JVM or CPU. It's only available in Xeon Phi, a HPC accelerator card with many slow cores.

sargun 9y ago
AVX and AVX2 are also included.

zbjornson 9y ago

CPUs throttle down for AVX and AVX2 workloads as well. Hopefully your vectorization is sufficient to make up for the throttling.