3 ms·

Well if by huge you mean 10ish microseconds, then yeah. There is also hysteresis built in so the transitions can only occur about once per millisecond. This put

Well if by huge you mean 10ish microseconds, then yeah. There is also hysteresis built in so the transitions can only occur about once per millisecond. This puts a hard bound on the cost of the transitions themselves of about 1%. So no, I don't think the transitions are a big problem for any throughput case, although if you are counting microseconds for some type of low latency thing then sure it can matter. The bigger problem is usually the frequency reduction.

Twirrim 8y ago
The amount of impact is heavily dependent on the class of CPU you use from Intel (gee, thanks Intel. Awesome way to make optimising code a pain in the arse). Bronze you probably really don't want to do AVX512 unless it's all heavily AVX512. Silver is better, but not great (which is what Cloudflare were using and ran in to). Gold or Platinum you're not likely to see much degradation from using AVX-512, even sparsely.
- BeeOnRope 8y ago
  Are you talking specifically about the transition penalties which I was referring to? I am not aware of any strong variance in this behavior based on CPU class, and I would be surprised if it existed. I've always measured around 8 to 10 us for these transitions (there may be both a real time and "clock cycles" component to the transition time so it might vary a bit when measured in real time based on the CPU frequency). It sounds to be like you are talking more about AVX-512 downclocking. In any case, if your algorithm can be make heavy and effective use of AVX-512 it's going to be a win on almost any class of chip since you are looking at something like a 2x baseline speedup and more in many cases. If not, then yeah, proceed with caution.
  - Twirrim 8y ago
    Yes, sorry, I'm referencing the down-clocking, which has an impact on other instructions that run. What Cloudflare were running in to was that AVX512 optimisations were only available on certain ciphers in OpenSSL, and weren't making up the majority of calls. They made up enough to be continually punting the chips down in to a lower core speed and significantly impacted other non-AVX instructions running on the system. Cloudflare ran in to this in part because they were opting for cheaper Silver class chips that are more likely to down-clock than Gold or Platinum. Bronze is worse. Contrast: Bronze: https://en.wikichip.org/wiki/intel/xeon_bronze/3106#Frequencies https://en.wikichip.org/wiki/intel/xeon_bronze/3106#Frequenc... Silver: https://en.wikichip.org/wiki/intel/xeon_silver/4108#Frequencies https://en.wikichip.org/wiki/intel/xeon_silver/4108#Frequenc... Gold: https://en.wikichip.org/wiki/intel/xeon_gold/5118#Frequencies https://en.wikichip.org/wiki/intel/xeon_gold/5118#Frequencie... Platinum: https://en.wikichip.org/wiki/intel/xeon_platinum/8153#Frequencies https://en.wikichip.org/wiki/intel/xeon_platinum/8153#Freque...
    - BeeOnRope 8y ago
      Right, downclocking is a different matter and I'm aware of the pitfalls there. As you point out, the extent of the downclocking varies based on the model - and also the exact "license" chosen depends not only on the instructions used, but their relative intensity (e.g., you can do a lot of 512-bit FMA instruction without suffering the slowest speeds, but once FMAs are dense enough you'll get the downclocking). FWIW, I recently tested a consumer Cannonlake chip and there is no downclocking even with heavy AVX-512: all types of tested code ran at the full turbo frequency. So it seems that not all chips will have these varying frequencies based on ISA extensions used.
      - Twirrim 8y ago
        Oh wow. That makes this even more of a mess than I'd realised. This seems an optimisation nightmare. You'd need to know a lot about the runtime and running environment to be able to figure out whether or not to enable AVX-512 instructions. Seems like something only JIT runtimes would be able to realistically handle on the fly.
deleted 8y ago
[deleted]

Twirrim 8y ago

The amount of impact is heavily dependent on the class of CPU you use from Intel (gee, thanks Intel. Awesome way to make optimising code a pain in the arse). Bronze you probably really don't want to do AVX512 unless it's all heavily AVX512. Silver is better, but not great (which is what Cloudflare were using and ran in to). Gold or Platinum you're not likely to see much degradation from using AVX-512, even sparsely.

BeeOnRope 8y ago
Are you talking specifically about the transition penalties which I was referring to? I am not aware of any strong variance in this behavior based on CPU class, and I would be surprised if it existed. I've always measured around 8 to 10 us for these transitions (there may be both a real time and "clock cycles" component to the transition time so it might vary a bit when measured in real time based on the CPU frequency). It sounds to be like you are talking more about AVX-512 downclocking. In any case, if your algorithm can be make heavy and effective use of AVX-512 it's going to be a win on almost any class of chip since you are looking at something like a 2x baseline speedup and more in many cases. If not, then yeah, proceed with caution.
- Twirrim 8y ago
  Yes, sorry, I'm referencing the down-clocking, which has an impact on other instructions that run. What Cloudflare were running in to was that AVX512 optimisations were only available on certain ciphers in OpenSSL, and weren't making up the majority of calls. They made up enough to be continually punting the chips down in to a lower core speed and significantly impacted other non-AVX instructions running on the system. Cloudflare ran in to this in part because they were opting for cheaper Silver class chips that are more likely to down-clock than Gold or Platinum. Bronze is worse. Contrast: Bronze: https://en.wikichip.org/wiki/intel/xeon_bronze/3106#Frequencies https://en.wikichip.org/wiki/intel/xeon_bronze/3106#Frequenc... Silver: https://en.wikichip.org/wiki/intel/xeon_silver/4108#Frequencies https://en.wikichip.org/wiki/intel/xeon_silver/4108#Frequenc... Gold: https://en.wikichip.org/wiki/intel/xeon_gold/5118#Frequencies https://en.wikichip.org/wiki/intel/xeon_gold/5118#Frequencie... Platinum: https://en.wikichip.org/wiki/intel/xeon_platinum/8153#Frequencies https://en.wikichip.org/wiki/intel/xeon_platinum/8153#Freque...
  - BeeOnRope 8y ago
    Right, downclocking is a different matter and I'm aware of the pitfalls there. As you point out, the extent of the downclocking varies based on the model - and also the exact "license" chosen depends not only on the instructions used, but their relative intensity (e.g., you can do a lot of 512-bit FMA instruction without suffering the slowest speeds, but once FMAs are dense enough you'll get the downclocking). FWIW, I recently tested a consumer Cannonlake chip and there is no downclocking even with heavy AVX-512: all types of tested code ran at the full turbo frequency. So it seems that not all chips will have these varying frequencies based on ISA extensions used.
    - Twirrim 8y ago
      Oh wow. That makes this even more of a mess than I'd realised. This seems an optimisation nightmare. You'd need to know a lot about the runtime and running environment to be able to figure out whether or not to enable AVX-512 instructions. Seems like something only JIT runtimes would be able to realistically handle on the fly.

deleted 8y ago

[deleted]