3 ms·

Now that major cloud vendors are selling VMs with guaranteed AVX512 support, how are they going to deal with the "noisy neighbor" problem?

by stu2010 9y ago

desdiv 9y ago
They don't? AFAIK no cloud vendor guarantees consistent performance on their multi-tenancy offerings (at least none backed by SLAs). Vendors sell both dedicated tenancy, with consistent performance, at a higher price and shared tenancy, with variable performance, at a lower price and expect customers who choose the latter to deal with the variance on their own. Large companies run a benchmark after each instance startup to check how noisy that instance's neighbors are. If it's bad then they simply terminate the instance and start another one[0]. [0] https://www.reddit.com/r/aws/comments/547xbx/netflix_found_5x_performance_variation_between/ https://www.reddit.com/r/aws/comments/547xbx/netflix_found_5...
microcolonel 9y ago
Giant heatsinks, and more sockets! More likely though, they'll continue not to care. Performance is already hideously variable, it'll just get a bit worse.
sargun 9y ago
For one, this is done on a core-by-core level. I believe Intel has the capability to monitor for this behaviour and do some dynamic throttling in certain models, which cloud vendors may have access to. I think they introduced some of this in Broadwell-E, where you can set the AVX offset per core, and the on-die PCU will throttle that core below the AVX base frequency, allowing the rest of the cores to remain at speed. These are typically controlled by BIOS, or MSRs. We're currently trying to figure out how to do deal with this. Some ideas come from Google's CPI2 paper, and trying to dynamically schedule workloads with diversity if we think they interfere. Other thoughts have been simpler, like core pinning (knapsacking for latency, or throughput). this is hard. Disclaimer: These views represent my own, and not my employer's, or their vendor's views

Now that major cloud vendors are selling VMs with guaranteed AVX512 support, how are they going to deal with the "noisy neighbor" problem?

desdiv 9y ago

They don't? AFAIK no cloud vendor guarantees consistent performance on their multi-tenancy offerings (at least none backed by SLAs). Vendors sell both dedicated tenancy, with consistent performance, at a higher price and shared tenancy, with variable performance, at a lower price and expect customers who choose the latter to deal with the variance on their own. Large companies run a benchmark after each instance startup to check how noisy that instance's neighbors are. If it's bad then they simply terminate the instance and start another one[0]. [0] https://www.reddit.com/r/aws/comments/547xbx/netflix_found_5x_performance_variation_between/ https://www.reddit.com/r/aws/comments/547xbx/netflix_found_5...

microcolonel 9y ago

Giant heatsinks, and more sockets! More likely though, they'll continue not to care. Performance is already hideously variable, it'll just get a bit worse.

sargun 9y ago

For one, this is done on a core-by-core level. I believe Intel has the capability to monitor for this behaviour and do some dynamic throttling in certain models, which cloud vendors may have access to. I think they introduced some of this in Broadwell-E, where you can set the AVX offset per core, and the on-die PCU will throttle that core below the AVX base frequency, allowing the rest of the cores to remain at speed. These are typically controlled by BIOS, or MSRs. We're currently trying to figure out how to do deal with this. Some ideas come from Google's CPI2 paper, and trying to dynamically schedule workloads with diversity if we think they interfere. Other thoughts have been simpler, like core pinning (knapsacking for latency, or throughput). this is hard. Disclaimer: These views represent my own, and not my employer's, or their vendor's views