I was just talking about this the other day on reddit. That situation was overblown. Cloudflare were cheaping out, and using chips that _weren't_ recommended

I was just talking about this the other day on reddit. That situation was overblown. Cloudflare were cheaping out, and using chips that _weren't_ recommended or designed for use with AVX-512, and then being surprised that they weren't getting good performance out of it. Cloudflare's chip where they saw the problems, was the Xeon Silver 4116: https://en.wikichip.org/wiki/intel/xeon_silver/4116#Frequencies https://en.wikichip.org/wiki/intel/xeon_silver/4116#Frequenc... Note that for most AVX workloads, it's fine. AVX2 sees a very minimal clock speed drop until almost all cores are actively engaged in the work. It's also worth mentioning that since Haswell days, the CPU groups cores doing AVX work together, to reduce the likelihood of the downclock impacting non-AVX work (I am somewhat curious what the impact on L1 / L2 caching is from that). AVX-512 is where it can hurt, but it really depends on the workload. Lots of AVX-512 instructions and you're fine, the throughput of the AVX-512 instructions is higher than scalar or AVX2 instructions, even with the down-clock. The important thing to note here, is that Cloudflare went with what amounts to a bargain basement server CPU. It's almost on the absolute bottom end of the range, and it's a chip not designed for AVX workloads (or indeed anything resembling high performance work). Just take a look at the product brief from when the family was launched (page 8): https://www.intel.com/content/dam/www/public/us/en/documents/product-briefs/xeon-scalable-platform-brief.pdf https://www.intel.com/content/dam/www/public/us/en/documents... If they'd actually taken a chip range designed for the workload, even at the bottom end of that range, just $100 or so more than the one they did choose: https://en.wikichip.org/wiki/intel/xeon_gold/5115#Frequencies https://en.wikichip.org/wiki/intel/xeon_gold/5115#Frequencie... Notice that it can handle a fair level of AVX-512 instructions before it down-clocks, and even then it takes a while before the down-clocking amount is significant, and it can handle significant AVX2 workloads before the maximum frequency gets affected at all (at the point where AVX2 starts causing it to down=clock, you'd be more than reaping the benefits of the faster instructions). For just a few hundred more dollars: https://en.wikichip.org/wiki/intel/xeon_gold/6126#Frequencies https://en.wikichip.org/wiki/intel/xeon_gold/6126#Frequencie..., you can be utilising all the cores doing AVX-512 instructions and still be faster than the not-designed-for-the-workload chip that Cloudflare cheaped out on. Note: These extra costs on CPUs are negligible when you take in to account depreciation, server lifetime etc. The biggest lifetime cost of a rack of servers in production is never the cost of the servers. It's your other OpEx. Cheaping out on server processors is a perfect example of what AWS employees call "Frupidity": Stupidity through Frugality. (Frugality is a leadership value in AWS, and sometimes it leads to the most phenomenally stupid things happening) Shift up in to the slightly more premium range, towards the kinds of chips you're likely to be running on in cloud platforms: https://en.wikichip.org/wiki/intel/xeon_platinum/8124#Frequencies https://en.wikichip.org/wiki/intel/xeon_platinum/8124#Freque... You can be using all cores with AVX-512 instructions, and see a drop of only 300Mhz, on the entry level platinum chip from that same year as Cloudflare's chip.