7 ms·

AMD is in a really good position right now. Intel still has the lead in low idle power which is good in laptops. Ryzen lets different cores have different max

AMD is in a really good position right now. Intel still has the lead in low idle power which is good in laptops. Ryzen lets different cores have different max frequencies so if your code is single threaded and your operating system isn't the newest that could be a reason to go Intel. Likewise if its single threaded and can take advantage of AVX-512 or the Intel Math Kernel Library. But otherwise?

Jonnax 7y ago
I hear a lot about AVX-512 being really good. Is there any software that's commonly used that has a measurable performance boost with it? Or is it more specialised stuff?
- devonkim 7y ago
  Newer versions of the x265 encoder for h.265 / HEVC video standard get significant speed-ups. I believe Handbrake was recently updated for it.
  - av3csr 7y ago
    I think RPCS3 also uses it in the JIT
- KingMachiavelli 7y ago
  From what I've read on AVX-512, the big disadvantage is that the AVX-512 instructions are very CPU intensive so the max clock speeds is reduced if you heavily use AVX-512 instructions. Another disadvantage is that you have to recompile your code to use AVX512 but it seems general enough that compilers will use the instructions (to an extent) without specialized code [1]. [1] https://www.phoronix.com/scan.php?page=news_item&px=GCC-8-AVX-512-Benchmarks https://www.phoronix.com/scan.php?page=news_item&px=GCC-8-AV...
  - robocat 7y ago
    The problem appears to be that whether you get a performance increase or a performance penalty depends on the duty cycle of your AVX512 instructions. What is especially deceiving is profiling a function in a loop for more than say 50ms, when the normal function execution takes say 0.5ms. Long running functions get the most gain, while short running functions cause the most pain. That is because downclocking AVX512 lasts 2ms (with a 0.5ms setup). Certain instruction mixes will cause a general slowdown (10% degradation measured by CloudFlare under actual usage) even though the test profiling might predicts a performance gain. Single AVX512 instructions when the CPU is running at full speed have a counterintuitive perverse performance penalty - apparently running 4x slower than when the CPU changes to the slower L1 or L2 clocks. Sustained AVX512 usage has predictable performance. “ Intel made more aggressive use of AVX-512 instructions in earlier versions of the icc compiler, but has since removed most use unless the user asks for it with a special command line option.” is a strong indication that you need to be very careful about where you use the instructions. Running an encoder for 1 second - likely candidate. Occasional 1ms functions or single AVX512 instructions on a web server - likely penalty.
- dragontamer 7y ago
  > I hear a lot about AVX-512 being really good. Its a great instruction set, Absolutely great, AVX512 supports gather/scatter, a whole slew of efficient processing instructions, etc. etc. However, AVX512 has poor implementations right now. Skylake-X is one of the only implementations, and running it drops the clock-rate in ways that are difficult to predict. (One core running AVX512 drops the clock of other cores, slowing down the throughput of the entire server). Traditionally, the first implementation of these instruction sets always a degree of "emulated". For example, the gather/scatter instructions aren't much faster than load/stores in practice. So while the AVX512 instruction set could theoretically be efficiently implemented, it seem like Skylake-X's implementation leaves much to be desired. Hopefully future implementations will be better. ------ The other major implementation of AVX512 is Xeon Phi, which has been deemed end-of-life. I like the idea of Xeon Phi, but it just didn't seem to work out in practice.
  - gameswithgo 7y ago
    the clock rate has nothing to do with a bad implementation, all the computation just makes a lot of heat. the performance increase is still massive, often more than 2x avx, which also throttles btw
    - DuskStar 7y ago
      The clock rate issue isn't the fact that it downclocks a core when moving to AVX512, it's that it downclocked all the other cores on the processor at the same time.
      - robocat 7y ago
        From what I have read, AVX512 only affects the one core (downclocking License level L1 or L2), it is older CPUs with AVX2 that affected all cores. Independently thermal throttling can occur which would affect all cores, although presumably the CPU is generating heat per numeric operation, so AVX512 is neutral versus other instructions per numeric operation.
        7y ago
- 3JPLW 7y ago
  One challenge with AVX-512 is that it can actually _slow down_ your code. It's so power hungry that if you're using it on more than one core it almost immediately incurs significant throttling. Now, if everything you're doing is 512 bits at a time, you're still winning. But if you're interleaving scalar and vector arithmetic, the drop in clock speeds could slow down the scalar code quite substantially. https://blog.cloudflare.com/on-the-dangers-of-intels-frequency-scaling/ https://blog.cloudflare.com/on-the-dangers-of-intels-frequen...
  - semi-extrinsic 7y ago
    How fast does the CPU step up and down the throttling caused by AVX-512? Basically, if you are interleaving like you suggest, does the processor detect this and reduce the throttling by the "duty cycle" of 512-bit operations? If not, could there be a way to tell the CPU to do this?
    - PeCaN 7y ago
      >How fast does the CPU step up and down the throttling caused by AVX-512? It's actually really, really slow. On newer (I think around Skylake-X, which is when AVX-512 was introduced) CPUs it takes up to 500 microseconds i.e. millions of cycles to activate AVX-512. This can't really be made faster because they actually need to give the voltage regulators time to adjust or the chip literally brows out. During this time AVX-512 instructions execute on the AVX-256 datapath¹. Once AVX-512 is activated the clock of that core is reduced by about 25% and it starts a 2ms timer which is reset whenever another AVX-512 instruction is issued. AFAIK Intel doesn't say how long it takes to raise the frequency again once the timer expires. (This is something of a simplification because there are actually two AVX power licenses, the first allowing AVX-256 and a limited set² of AVX-512 instructions, reducing clock by about 15%, and the second allowing everything. Also, executing a single AVX-512 instruction doesn't immediately request a higher power license, you have to execute a certain number of them.) ― This is actually the better version. On Haswell executing any AVX-256 instruction would reduce the frequency of every core by about 15-20%. But hey, at least it only takes about 150k cycles to activate (not much of a consolation, I know). Beats me how long it stays throttled for. (I don't know what exactly Broadwell did. I don't think it throttled all cores, but it didn't have the additional power license with reduced throttling that Skylake has.) ― ¹ Or the 128-bit datapath if the core is at the lowest power license (which still lets you use 128-bit SSE instructions, and basic AVX-256 instructions). ² Basically anything that doesn't execute on the floating-point unit, which means no floating point and no integer multiplication (which uses the FPU). This is actually kinda the saving grace of the whole thing, since it means you can vectorize things like memcmp and strlen without requiring the highest power license.
- pkaye 7y ago
  I've used it once but on Intel the clock is throttled with AVX-512 so the overall program performance improved only 1% because the non-SIMD code was running at a slower clock.
ogrisel 7y ago
The BLAS & Lapack subset of the API of the Intel Math Kernel Library (MKL) is very well implemented in open source projects such as OpenBLAS and BLIS: https://github.com/flame/blis https://github.com/flame/blis Both are well optimized for AMD CPUs.
- vj44 7y ago
  I work in this space... and let's just say that MKL is definitely NOT well optimized for AMD's chips. You'll be lucky to get 10-20% efficiency. Nevermind openblas.
  - localhost 7y ago
    Intel optimizes these libraries down to the stepping level of the CPUs. So not surprising if they are not optimized at all for AMD
    - jejones3141 7y ago
      Is it anything like the way their compiler detected SSEn in a way that guaranteed it wouldn't use those instructions on AMD processors even if they supported them?
      - tanderson92 7y ago
        Of course. It's very much intentional, and "not optimized for AMD" is putting it very very mildly. They don't need to optimize purely for stepping level, they could provide sane codepaths for when the CPU flags indicate certain features. See my other comment on this topic.
      - eyegor 7y ago
        Yes but it wasn't a (if not amd). It was a series of checks based on specific families of Intel cpus, such as haswell, sandy bridge, etc. So it was never actually querying whether the cpu supported instruction x, it was asking what family it belonged to and then applying static rules based on that. Maybe nuance, but it also has the potential to hurt their processors if not kept up on so maybe less malice and more convenience?
dragontamer 7y ago
The main selling point for Intel isn't speed anymore. Intel still has superior performance counters and debugging features. Mozilla's rr (Record and Replay framework) only works on Intel for example, and Intel vTune is a very good tool. AVX512 is also an advantage, as you've noted. There are other instruction set advantages: I think Intel has faster division / modulus operator, and also has single-clock pext / pdep (useful in chess programs). For most people however, who might not be using those tools, I'd argue that AMD's offerings are superior.
- rbanffy 7y ago
  Even if you do, in most cases you won't use them on all your workloads. This means not all your boxes need to be Intel.
  - dragontamer 7y ago
    I would hope that your production hardware matches your developer / staging / testing hardware. Lets say production is 50% slower than what's tested in staging / developer test cases. Is it the data in production that causes this performance loss? Or is it hardware differences? If you are using Intel tools to debug performance problems on developer / testing stages, you probably want to keep using those Intel tools in staging / production. There are enough cache differences and instruction-level differences (speed of "division" instruction. PEXT vs PDEP. Cache differenches, branch predictor differences, TLB differences) between the chips. Intel has interesting optimizations: an Intel Ethernet card drops the data off in L3 cache (bypassing DDR4 RAM entirely). These little differences in the driver / motherboard / CPU can have a huge difference in performance, and complicate performance testing / performance debugging. If you are deploying to AMD hardware for production, you probably want to be running AMD hardware in testing / developer stages as well. You want all your hardware performing as similarly as possible.
    - AstralStorm 7y ago
      Said interesting networking optimization is a gaping security home that has already been exploited in the wild.
Mathnerd314 7y ago
I looked for some numbers on low idle power, apparently 3k significantly improved something: https://www.reddit.com/r/AMD_Stock/comments/ado0ix/ryzen_mobile_idle_power_consumption_seems_to_have/ https://www.reddit.com/r/AMD_Stock/comments/ado0ix/ryzen_mob.... This reviewer seems impressed, something about only using 10W in idle on desktop: https://youtu.be/M5pHUHGZ7hU?t=363 https://youtu.be/M5pHUHGZ7hU?t=363. And a head-to-head test of a laptop available in AMD and Intel variants says it has better battery life, although the screen panel could be the reason: https://www.notebookcheck.net/Lenovo-ThinkPad-T495-Review-business-laptop-with-AMD-processor-long-battery-life-and-good-display.434716.0.html https://www.notebookcheck.net/Lenovo-ThinkPad-T495-Review-bu... I'd like more info, and Intel still probably has a lot of firmware tweaks etc. that AMD has to implement to win microbenchmarks, but to a first order it's not clear Intel has a lead there anymore.
zelon88 7y ago
I think most server operators look at overall performance. Once you start buying hardware specifically for one purpose you're cornering yourself. Besides, who spends $15,000 on a mid-high end server to run single threaded applications anyway?
- eyegor 7y ago
  I happen to know of several companies doing physics problems that scale poorly across cores that spend far north of that, usually building out small clusters. Then you run 100s of independent simulations since each individual one doesn't really scale.
  - JohnJamesRambo 7y ago
    What are the physics problems?
    - eyegor 7y ago
      Fluid flow and most particle simulations with a large number of particles. The limiting factor is the inter particle interactions, so all the calculations have to feed back into each other.
      - BubRoss 7y ago
        Both of those problems are well worn and can scale to as many cores as we can put in a single computer. Whether it is a navier-stokes grid/image fluid simulation, arbitrary points in space that work off of nearest neighbors or a combination of both (by rasterizing into a grid and using that to move the particles), there are many straightforward ways to use lots of CPUs. Fork join parallelism is a start. Sorting particles into a kd-tree is done by recursively partitioning and the partitions can be distributed amount cores. The sorting structure can be read but not written by as many cores as you want, and thus their neighbors can be searched and found by all cores at once.
  - gameswithgo 7y ago

Jonnax 7y ago

I hear a lot about AVX-512 being really good. Is there any software that's commonly used that has a measurable performance boost with it? Or is it more specialised stuff?

devonkim 7y ago
Newer versions of the x265 encoder for h.265 / HEVC video standard get significant speed-ups. I believe Handbrake was recently updated for it.
- av3csr 7y ago
  I think RPCS3 also uses it in the JIT
KingMachiavelli 7y ago
From what I've read on AVX-512, the big disadvantage is that the AVX-512 instructions are very CPU intensive so the max clock speeds is reduced if you heavily use AVX-512 instructions. Another disadvantage is that you have to recompile your code to use AVX512 but it seems general enough that compilers will use the instructions (to an extent) without specialized code [1]. [1] https://www.phoronix.com/scan.php?page=news_item&px=GCC-8-AVX-512-Benchmarks https://www.phoronix.com/scan.php?page=news_item&px=GCC-8-AV...
- robocat 7y ago
  The problem appears to be that whether you get a performance increase or a performance penalty depends on the duty cycle of your AVX512 instructions. What is especially deceiving is profiling a function in a loop for more than say 50ms, when the normal function execution takes say 0.5ms. Long running functions get the most gain, while short running functions cause the most pain. That is because downclocking AVX512 lasts 2ms (with a 0.5ms setup). Certain instruction mixes will cause a general slowdown (10% degradation measured by CloudFlare under actual usage) even though the test profiling might predicts a performance gain. Single AVX512 instructions when the CPU is running at full speed have a counterintuitive perverse performance penalty - apparently running 4x slower than when the CPU changes to the slower L1 or L2 clocks. Sustained AVX512 usage has predictable performance. “ Intel made more aggressive use of AVX-512 instructions in earlier versions of the icc compiler, but has since removed most use unless the user asks for it with a special command line option.” is a strong indication that you need to be very careful about where you use the instructions. Running an encoder for 1 second - likely candidate. Occasional 1ms functions or single AVX512 instructions on a web server - likely penalty.
dragontamer 7y ago
> I hear a lot about AVX-512 being really good. Its a great instruction set, Absolutely great, AVX512 supports gather/scatter, a whole slew of efficient processing instructions, etc. etc. However, AVX512 has poor implementations right now. Skylake-X is one of the only implementations, and running it drops the clock-rate in ways that are difficult to predict. (One core running AVX512 drops the clock of other cores, slowing down the throughput of the entire server). Traditionally, the first implementation of these instruction sets always a degree of "emulated". For example, the gather/scatter instructions aren't much faster than load/stores in practice. So while the AVX512 instruction set could theoretically be efficiently implemented, it seem like Skylake-X's implementation leaves much to be desired. Hopefully future implementations will be better. ------ The other major implementation of AVX512 is Xeon Phi, which has been deemed end-of-life. I like the idea of Xeon Phi, but it just didn't seem to work out in practice.
- gameswithgo 7y ago
  the clock rate has nothing to do with a bad implementation, all the computation just makes a lot of heat. the performance increase is still massive, often more than 2x avx, which also throttles btw
  - DuskStar 7y ago
    The clock rate issue isn't the fact that it downclocks a core when moving to AVX512, it's that it downclocked all the other cores on the processor at the same time.
    - robocat 7y ago
      From what I have read, AVX512 only affects the one core (downclocking License level L1 or L2), it is older CPUs with AVX2 that affected all cores. Independently thermal throttling can occur which would affect all cores, although presumably the CPU is generating heat per numeric operation, so AVX512 is neutral versus other instructions per numeric operation.
      - 7y ago
3JPLW 7y ago
One challenge with AVX-512 is that it can actually _slow down_ your code. It's so power hungry that if you're using it on more than one core it almost immediately incurs significant throttling. Now, if everything you're doing is 512 bits at a time, you're still winning. But if you're interleaving scalar and vector arithmetic, the drop in clock speeds could slow down the scalar code quite substantially. https://blog.cloudflare.com/on-the-dangers-of-intels-frequency-scaling/ https://blog.cloudflare.com/on-the-dangers-of-intels-frequen...
- semi-extrinsic 7y ago
  How fast does the CPU step up and down the throttling caused by AVX-512? Basically, if you are interleaving like you suggest, does the processor detect this and reduce the throttling by the "duty cycle" of 512-bit operations? If not, could there be a way to tell the CPU to do this?
  - PeCaN 7y ago
    >How fast does the CPU step up and down the throttling caused by AVX-512? It's actually really, really slow. On newer (I think around Skylake-X, which is when AVX-512 was introduced) CPUs it takes up to 500 microseconds i.e. millions of cycles to activate AVX-512. This can't really be made faster because they actually need to give the voltage regulators time to adjust or the chip literally brows out. During this time AVX-512 instructions execute on the AVX-256 datapath¹. Once AVX-512 is activated the clock of that core is reduced by about 25% and it starts a 2ms timer which is reset whenever another AVX-512 instruction is issued. AFAIK Intel doesn't say how long it takes to raise the frequency again once the timer expires. (This is something of a simplification because there are actually two AVX power licenses, the first allowing AVX-256 and a limited set² of AVX-512 instructions, reducing clock by about 15%, and the second allowing everything. Also, executing a single AVX-512 instruction doesn't immediately request a higher power license, you have to execute a certain number of them.) ― This is actually the better version. On Haswell executing any AVX-256 instruction would reduce the frequency of every core by about 15-20%. But hey, at least it only takes about 150k cycles to activate (not much of a consolation, I know). Beats me how long it stays throttled for. (I don't know what exactly Broadwell did. I don't think it throttled all cores, but it didn't have the additional power license with reduced throttling that Skylake has.) ― ¹ Or the 128-bit datapath if the core is at the lowest power license (which still lets you use 128-bit SSE instructions, and basic AVX-256 instructions). ² Basically anything that doesn't execute on the floating-point unit, which means no floating point and no integer multiplication (which uses the FPU). This is actually kinda the saving grace of the whole thing, since it means you can vectorize things like memcmp and strlen without requiring the highest power license.
pkaye 7y ago
I've used it once but on Intel the clock is throttled with AVX-512 so the overall program performance improved only 1% because the non-SIMD code was running at a slower clock.

ogrisel 7y ago

The BLAS & Lapack subset of the API of the Intel Math Kernel Library (MKL) is very well implemented in open source projects such as OpenBLAS and BLIS: https://github.com/flame/blis https://github.com/flame/blis Both are well optimized for AMD CPUs.

vj44 7y ago
I work in this space... and let's just say that MKL is definitely NOT well optimized for AMD's chips. You'll be lucky to get 10-20% efficiency. Nevermind openblas.
- localhost 7y ago
  Intel optimizes these libraries down to the stepping level of the CPUs. So not surprising if they are not optimized at all for AMD
  - jejones3141 7y ago
    Is it anything like the way their compiler detected SSEn in a way that guaranteed it wouldn't use those instructions on AMD processors even if they supported them?
    - tanderson92 7y ago
      Of course. It's very much intentional, and "not optimized for AMD" is putting it very very mildly. They don't need to optimize purely for stepping level, they could provide sane codepaths for when the CPU flags indicate certain features. See my other comment on this topic.
    - eyegor 7y ago
      Yes but it wasn't a (if not amd). It was a series of checks based on specific families of Intel cpus, such as haswell, sandy bridge, etc. So it was never actually querying whether the cpu supported instruction x, it was asking what family it belonged to and then applying static rules based on that. Maybe nuance, but it also has the potential to hurt their processors if not kept up on so maybe less malice and more convenience?

dragontamer 7y ago

The main selling point for Intel isn't speed anymore. Intel still has superior performance counters and debugging features. Mozilla's rr (Record and Replay framework) only works on Intel for example, and Intel vTune is a very good tool. AVX512 is also an advantage, as you've noted. There are other instruction set advantages: I think Intel has faster division / modulus operator, and also has single-clock pext / pdep (useful in chess programs). For most people however, who might not be using those tools, I'd argue that AMD's offerings are superior.

rbanffy 7y ago
Even if you do, in most cases you won't use them on all your workloads. This means not all your boxes need to be Intel.
- dragontamer 7y ago
  I would hope that your production hardware matches your developer / staging / testing hardware. Lets say production is 50% slower than what's tested in staging / developer test cases. Is it the data in production that causes this performance loss? Or is it hardware differences? If you are using Intel tools to debug performance problems on developer / testing stages, you probably want to keep using those Intel tools in staging / production. There are enough cache differences and instruction-level differences (speed of "division" instruction. PEXT vs PDEP. Cache differenches, branch predictor differences, TLB differences) between the chips. Intel has interesting optimizations: an Intel Ethernet card drops the data off in L3 cache (bypassing DDR4 RAM entirely). These little differences in the driver / motherboard / CPU can have a huge difference in performance, and complicate performance testing / performance debugging. If you are deploying to AMD hardware for production, you probably want to be running AMD hardware in testing / developer stages as well. You want all your hardware performing as similarly as possible.
  - AstralStorm 7y ago
    Said interesting networking optimization is a gaping security home that has already been exploited in the wild.

Mathnerd314 7y ago

I looked for some numbers on low idle power, apparently 3k significantly improved something: https://www.reddit.com/r/AMD_Stock/comments/ado0ix/ryzen_mobile_idle_power_consumption_seems_to_have/ https://www.reddit.com/r/AMD_Stock/comments/ado0ix/ryzen_mob.... This reviewer seems impressed, something about only using 10W in idle on desktop: https://youtu.be/M5pHUHGZ7hU?t=363 https://youtu.be/M5pHUHGZ7hU?t=363. And a head-to-head test of a laptop available in AMD and Intel variants says it has better battery life, although the screen panel could be the reason: https://www.notebookcheck.net/Lenovo-ThinkPad-T495-Review-business-laptop-with-AMD-processor-long-battery-life-and-good-display.434716.0.html https://www.notebookcheck.net/Lenovo-ThinkPad-T495-Review-bu... I'd like more info, and Intel still probably has a lot of firmware tweaks etc. that AMD has to implement to win microbenchmarks, but to a first order it's not clear Intel has a lead there anymore.

zelon88 7y ago

I think most server operators look at overall performance. Once you start buying hardware specifically for one purpose you're cornering yourself. Besides, who spends $15,000 on a mid-high end server to run single threaded applications anyway?

eyegor 7y ago
I happen to know of several companies doing physics problems that scale poorly across cores that spend far north of that, usually building out small clusters. Then you run 100s of independent simulations since each individual one doesn't really scale.
- JohnJamesRambo 7y ago
  What are the physics problems?
  - eyegor 7y ago
    Fluid flow and most particle simulations with a large number of particles. The limiting factor is the inter particle interactions, so all the calculations have to feed back into each other.
    - BubRoss 7y ago
      Both of those problems are well worn and can scale to as many cores as we can put in a single computer. Whether it is a navier-stokes grid/image fluid simulation, arbitrary points in space that work off of nearest neighbors or a combination of both (by rasterizing into a grid and using that to move the particles), there are many straightforward ways to use lots of CPUs. Fork join parallelism is a start. Sorting particles into a kd-tree is done by recursively partitioning and the partitions can be distributed amount cores. The sorting structure can be read but not written by as many cores as you want, and thus their neighbors can be searched and found by all cores at once.
- gameswithgo 7y ago