4 ms·

AVX-512 causes frequency throttling for thermal reasons. https://lemire.me/blog/2018/08/25/avx-512-throttling-heavy-instructions-are-maybe-not-so-dangerous/ ht

AVX-512 causes frequency throttling for thermal reasons. https://lemire.me/blog/2018/08/25/avx-512-throttling-heavy-instructions-are-maybe-not-so-dangerous/ https://lemire.me/blog/2018/08/25/avx-512-throttling-heavy-i... https://lemire.me/blog/2018/08/24/trying-harder-to-make-avx-512-look-bad-my-quantified-and-reproducible-results/ https://lemire.me/blog/2018/08/24/trying-harder-to-make-avx-... https://lemire.me/blog/2018/08/15/the-dangers-of-avx-512-throttling-a-3-impact/ https://lemire.me/blog/2018/08/15/the-dangers-of-avx-512-thr... Any performance gain from fewer cycle cost may be lost from the CPU being slower. Maybe there are situations where AVX-512 is faster but how is one to reason about performance when some instructions can slow the CPU.

greggyb 6y ago
> but how is one to reason about performance when some instructions can slow the CPU. The same way one should always do so on an advanced out-of-order CPU largely being programmed in modern high level languages: through thorough profiling and performance testing.
- deleuze 6y ago
  Empiricism? In my software development? No way.
  - haecceity 6y ago
    I appreciate the humor but that's more time consuming than comparing instruction cycle counts and data dependencies. It's also workload dependent when it wouldn't be if they didn't come up with this throttling thing even if out of order execution makes it complicated.
    - greggyb 6y ago
      Most programmers are not regularly writing primarily assembly or reviewing the assembly generated by their compiler. These are the scenarios where instruction cycle counts become relevant - everything else is abstracted too far to rely on anything other than profiling and testing. If that's you, then I'm sorry for my response which didn't really take you into account.
bayindirh 6y ago
Also, Intel almost never provides the frequency throttling details for AVX-512 heavy workloads, which is called AVX base frequency. You can only see the performance values for yourself, or believe the SPEC benchmarks. If you're working in an HPC center, you may ask your fellow centers and get off-the-record answers, sometimes.
- tarlinian 6y ago
  This is absolutely not true. They are available for every publicly available SKU. https://www.intel.com/content/www/us/en/products/docs/processors/xeon/2nd-gen-xeon-scalable-spec-update.html https://www.intel.com/content/www/us/en/products/docs/proces...
  - bayindirh 6y ago
    When we got our processors last year for our new cluster (I work in an HPC center), Intel explicitly said that it's confidential information and cannot be shared. I remember reading the datasheets myself and not being able to find it, together with my teammates. So they changed their mind two months ago, and shared this info for their processors. Guess that the information lost its significance. Either way, a processor significantly throttles under AVX-512 load. Nice. Edit: I re-read the document and found our SKUs under the "newly added" table and, clarified the comment. Original Datasheets: - https://www.intel.com/content/www/tr/tr/products/docs/processors/xeon/2nd-gen-xeon-scalable-datasheet-vol-1.html https://www.intel.com/content/www/tr/tr/products/docs/proces... - https://www.intel.com/content/www/tr/tr/products/docs/processors/xeon/2nd-gen-xeon-scalable-datasheet-vol-2.html https://www.intel.com/content/www/tr/tr/products/docs/proces... Spec Update - For re-referencing: - https://www.intel.com/content/dam/www/public/us/en/documents/specification-updates/xeon-scalable-spec-update.pdf https://www.intel.com/content/dam/www/public/us/en/documents...
    - TinkersW 6y ago
      It has been publicly available since years ago(I remember reading the specification explaining how it worked), it was never a surprise like Cloudflare made it out to be.
solidasparagus 6y ago
You measure. It's actually not that hard - try with AVX and try without and see which is faster for your workloads.
corysama 6y ago
For the throttling operations you need to decide to commit to using them heavily or not. If you go all-in on them and use them well, you will get the same work done in less time and less joules than without them. If they won't help, don't use them. But, popping in and using them for tiny bits would be the slow and hot worst of both worlds.

greggyb 6y ago

> but how is one to reason about performance when some instructions can slow the CPU. The same way one should always do so on an advanced out-of-order CPU largely being programmed in modern high level languages: through thorough profiling and performance testing.

deleuze 6y ago
Empiricism? In my software development? No way.
- haecceity 6y ago
  I appreciate the humor but that's more time consuming than comparing instruction cycle counts and data dependencies. It's also workload dependent when it wouldn't be if they didn't come up with this throttling thing even if out of order execution makes it complicated.
  - greggyb 6y ago
    Most programmers are not regularly writing primarily assembly or reviewing the assembly generated by their compiler. These are the scenarios where instruction cycle counts become relevant - everything else is abstracted too far to rely on anything other than profiling and testing. If that's you, then I'm sorry for my response which didn't really take you into account.

bayindirh 6y ago

Also, Intel almost never provides the frequency throttling details for AVX-512 heavy workloads, which is called AVX base frequency. You can only see the performance values for yourself, or believe the SPEC benchmarks. If you're working in an HPC center, you may ask your fellow centers and get off-the-record answers, sometimes.

tarlinian 6y ago
This is absolutely not true. They are available for every publicly available SKU. https://www.intel.com/content/www/us/en/products/docs/processors/xeon/2nd-gen-xeon-scalable-spec-update.html https://www.intel.com/content/www/us/en/products/docs/proces...
- bayindirh 6y ago
  When we got our processors last year for our new cluster (I work in an HPC center), Intel explicitly said that it's confidential information and cannot be shared. I remember reading the datasheets myself and not being able to find it, together with my teammates. So they changed their mind two months ago, and shared this info for their processors. Guess that the information lost its significance. Either way, a processor significantly throttles under AVX-512 load. Nice. Edit: I re-read the document and found our SKUs under the "newly added" table and, clarified the comment. Original Datasheets: - https://www.intel.com/content/www/tr/tr/products/docs/processors/xeon/2nd-gen-xeon-scalable-datasheet-vol-1.html https://www.intel.com/content/www/tr/tr/products/docs/proces... - https://www.intel.com/content/www/tr/tr/products/docs/processors/xeon/2nd-gen-xeon-scalable-datasheet-vol-2.html https://www.intel.com/content/www/tr/tr/products/docs/proces... Spec Update - For re-referencing: - https://www.intel.com/content/dam/www/public/us/en/documents/specification-updates/xeon-scalable-spec-update.pdf https://www.intel.com/content/dam/www/public/us/en/documents...
  - TinkersW 6y ago
    It has been publicly available since years ago(I remember reading the specification explaining how it worked), it was never a surprise like Cloudflare made it out to be.

solidasparagus 6y ago

You measure. It's actually not that hard - try with AVX and try without and see which is faster for your workloads.

corysama 6y ago

For the throttling operations you need to decide to commit to using them heavily or not. If you go all-in on them and use them well, you will get the same work done in less time and less joules than without them. If they won't help, don't use them. But, popping in and using them for tiny bits would be the slow and hot worst of both worlds.