6 ms·

I am not talking about ease of use, but about the downclock. The other extensions do not trigger it, not even AVX256. With AVX512 is not always a win, and you

I am not talking about ease of use, but about the downclock. The other extensions do not trigger it, not even AVX256. With AVX512 is not always a win, and you don't even know until you try in particular hardware.

klodolph 6y ago
Different causes, similar consequences.
- innocenat 6y ago
  Porting SSE to AVX code (with equivalent instruction and proper vzeropper) will increase performance in most case (the only case where it can be slower, on top of my head, is on Sandy Bridge). The same is not true for AVX to AVX512.
  - google234123 6y ago
    AVX to AVX512 will "increase performance in most case"s. https://www.researchgate.net/figure/Speedup-from-AVX-512-over-AVX2-for-optimized-NESAP-codes-on-Cori-KNL_fig4_320499324 https://www.researchgate.net/figure/Speedup-from-AVX-512-ove...
  - Const-me 6y ago
    It will increase performance if you have sufficient amount of dense data on input. When that’s the case, especially if the numbers being crunched are 32-bit floats, there’s not much point of doing it on CPU at all, GPGPUs are way more efficient for such tasks. However, imagine sparse matrix * dense vector multiplication. If you rarely have more than 4 consecutive non-zero elements in rows of the input matrix, and large gaps between non-zero elements, moving from SSE to AVX or AVX512 will decrease the performance, you’ll be just wasting electricity multiplying by zeros.
  - tarlinian 6y ago
    So in some sense very similar to SKX behavior? The first iteration of the instruction implementation requires judicious use of instructions, while later implementations (this is something to be upset about...those "later implementations" should have been available quite some time ago). This is also ignoring the fact that none of these penalties come into play if you use the AVX512 instructions with 256-bit or 128-bit vectors. (This still has significant benefits due to the much nicer set of shuffles, dedicated mask registers, etc.)
jcranmer 6y ago
The 256-bit vector instructions do trigger a downclock, but not as severe as the AVX512 downclock.
- jfkebwjsbx 6y ago
  You are 100% right, it is AVX1 I was thinking about (which nowadays I am not sure if that has downclock or not either).
MaxBarraclough 6y ago
It also has a 'warm up' phase, if I understand correctly. https://news.ycombinator.com/item?id=17936810 https://news.ycombinator.com/item?id=17936810 https://www.agner.org/optimize/blog/read.php?i=415 https://www.agner.org/optimize/blog/read.php?i=415
- google234123 6y ago
  Why link to a 5 year old thread? There has to be more recent work.
  - tarlinian 6y ago
    There is more recent work. This blog post by Travis Downs is the most detailed analysis of transition behavior I've seen: https://travisdowns.github.io/blog/2020/01/17/avxfreq1.html https://travisdowns.github.io/blog/2020/01/17/avxfreq1.html For general guidelines on when to use AVX-512, this (older post) remains best guide I've seen: https://lemire.me/blog/2018/09/07/avx-512-when-and-how-to-use-these-new-instructions/ https://lemire.me/blog/2018/09/07/avx-512-when-and-how-to-us...
    - google234123 6y ago
      So, are programs that are compiled with those instructions faster or slower? In my experience they have been faster.
      - klodolph 6y ago
        Short answer: Yes, faster. Long answer: It depends, and you may be measuring the wrong thing. Among other things, it depends on the workload and the exact processor. You can find plenty of cases where AVX512 makes things faster. You can also find cases where the entire system slows down because it is running sections of AVX512 code here and there—apparently, for certain Intel processors, the processor needs to turn on top 256 bits of the register files and interconnects, and to get full speed for AVX512 it will alter the processor’s voltage and clock speed. This reduces the speed of other instructions and even other cores on the same die (which may be surprising). While the specifics may be new, the generalities seem familiar—it has long been true that a well-intentioned improvement to a small section of your code base can improve performance locally while degrading overall system performance. The code that you’re working on occupies a smaller and smaller slice of your performance metrics, and meanwhile, the whole system is slowing down. There are so many reasons that this can happen, dynamic frequency scaling with AVX512 is just one more reason.

klodolph 6y ago

Different causes, similar consequences.

innocenat 6y ago
Porting SSE to AVX code (with equivalent instruction and proper vzeropper) will increase performance in most case (the only case where it can be slower, on top of my head, is on Sandy Bridge). The same is not true for AVX to AVX512.
- google234123 6y ago
  AVX to AVX512 will "increase performance in most case"s. https://www.researchgate.net/figure/Speedup-from-AVX-512-over-AVX2-for-optimized-NESAP-codes-on-Cori-KNL_fig4_320499324 https://www.researchgate.net/figure/Speedup-from-AVX-512-ove...
- Const-me 6y ago
  It will increase performance if you have sufficient amount of dense data on input. When that’s the case, especially if the numbers being crunched are 32-bit floats, there’s not much point of doing it on CPU at all, GPGPUs are way more efficient for such tasks. However, imagine sparse matrix * dense vector multiplication. If you rarely have more than 4 consecutive non-zero elements in rows of the input matrix, and large gaps between non-zero elements, moving from SSE to AVX or AVX512 will decrease the performance, you’ll be just wasting electricity multiplying by zeros.
- tarlinian 6y ago
  So in some sense very similar to SKX behavior? The first iteration of the instruction implementation requires judicious use of instructions, while later implementations (this is something to be upset about...those "later implementations" should have been available quite some time ago). This is also ignoring the fact that none of these penalties come into play if you use the AVX512 instructions with 256-bit or 128-bit vectors. (This still has significant benefits due to the much nicer set of shuffles, dedicated mask registers, etc.)

jcranmer 6y ago

The 256-bit vector instructions do trigger a downclock, but not as severe as the AVX512 downclock.

jfkebwjsbx 6y ago
You are 100% right, it is AVX1 I was thinking about (which nowadays I am not sure if that has downclock or not either).

MaxBarraclough 6y ago

It also has a 'warm up' phase, if I understand correctly. https://news.ycombinator.com/item?id=17936810 https://news.ycombinator.com/item?id=17936810 https://www.agner.org/optimize/blog/read.php?i=415 https://www.agner.org/optimize/blog/read.php?i=415

google234123 6y ago
Why link to a 5 year old thread? There has to be more recent work.
- tarlinian 6y ago
  There is more recent work. This blog post by Travis Downs is the most detailed analysis of transition behavior I've seen: https://travisdowns.github.io/blog/2020/01/17/avxfreq1.html https://travisdowns.github.io/blog/2020/01/17/avxfreq1.html For general guidelines on when to use AVX-512, this (older post) remains best guide I've seen: https://lemire.me/blog/2018/09/07/avx-512-when-and-how-to-use-these-new-instructions/ https://lemire.me/blog/2018/09/07/avx-512-when-and-how-to-us...
  - google234123 6y ago
    So, are programs that are compiled with those instructions faster or slower? In my experience they have been faster.
    - klodolph 6y ago
      Short answer: Yes, faster. Long answer: It depends, and you may be measuring the wrong thing. Among other things, it depends on the workload and the exact processor. You can find plenty of cases where AVX512 makes things faster. You can also find cases where the entire system slows down because it is running sections of AVX512 code here and there—apparently, for certain Intel processors, the processor needs to turn on top 256 bits of the register files and interconnects, and to get full speed for AVX512 it will alter the processor’s voltage and clock speed. This reduces the speed of other instructions and even other cores on the same die (which may be surprising). While the specifics may be new, the generalities seem familiar—it has long been true that a well-intentioned improvement to a small section of your code base can improve performance locally while degrading overall system performance. The code that you’re working on occupies a smaller and smaller slice of your performance metrics, and meanwhile, the whole system is slowing down. There are so many reasons that this can happen, dynamic frequency scaling with AVX512 is just one more reason.