5 ms·

It's not that simple for computing. I heard that in Data Science Intel is still preferred because of better AVX support. There are also things like Intel MKL.

by f311a 6y ago

It's not that simple for computing. I heard that in Data Science Intel is still preferred because of better AVX support. There are also things like Intel MKL. A lot of software can use it when compiled on a user machine.

Const-me 6y ago
> is still preferred because of better AVX support AVX1 and AVX2 performance is on par. For instance, vmulpd AVX1 instruction is faster on AMD, 3 versus 4 cycles. vpaddd AVX2 instruction is same at 1 cycle latency. vfmadd132pd FMA instruction is slightly faster on Intel, 4 versus 5 cycles. Throughput is the same across these two. I was looking at AMD Zen2 versus Intel Ice Lake. Some Intel chips have AVX512. Still, many practical applications don’t need that amount of SIMD wideness, and these who do are often a good fit for GPGPUs. > There are also things like Intel MKL There’re vendor-agnostic equivalents like Eigen.
- f311a 6y ago
  For some computer vision tasks, Tensorflow is much faster when you have AVX512. Also https://www.intel.com/content/www/us/en/artificial-intelligence/posts/tensorflow-optimizations-intel-xeon-scalable-processor.html https://www.intel.com/content/www/us/en/artificial-intellige...
  - Const-me 6y ago
    These results were achieved on dual-socket Xeon E5 2699v4 (the architecture is 5 years old and has no AVX512, they optimized for AVX2) and on Xeon Phi 7250 (that thing does have AVX512 but that’s not a processor, a specialized accelerator with 68 cores). Also Tensorflow is awesome fit for GPGPUs and is usually way faster on them.
- DoofusOfDeath 6y ago
  IIUC, Intel uses the term "AVX-512" as an umbrella term, and different processors support different subsets of "AVX-512" instructions [0]. AFAIK this is a break from previous Intel nomenclature, where any processor supporting e.g. "SSE4.2" instructions was guaranteed to support all SSE4.2 instructions. I'm concerned that sometimes this causes confusion when talking about processor -- software compatibility. [0] https://en.wikipedia.org/wiki/AVX-512 https://en.wikipedia.org/wiki/AVX-512
  - loeg 6y ago
    I think the consideration GP was trying to make is that Zen (at least 1 and 2 — and I haven't heard otherwise for 3) do not support 512-bit wide AVX registers at all.
- singhrac 6y ago
  > There’re vendor-agnostic equivalents like Eigen. That's looking at the wrong layer of the hierarchy, I think. There are many open-source linear algebra libraries, but iirc they all link against something that has a BLAS/LAPACK API. That might be something like MKL, OpenBLAS, ATLAS, etc. When I last checked, MKL was much faster than its competitors, and is only available (at full speed) on Intel CPUs. Has that changed?
  - Const-me 6y ago
    > iirc they all link against something that has a BLAS/LAPACK API Eigen can consume these I think, but they are optional. It has it’s own implementation of these, written in manually vectorized C++, with intrinsics, up to and including AVX512 (controlled with macros). For parallelization it uses OpenMP provided by the compiler (also controlled with a macro). > Has that changed? It’s hard to directly compare Eigen to the rest of them. They don’t do the same thing. One feature of Eigen is lazy evaluation. Expressions like a+b or a×b don’t return another matrix or vector; they return a placeholder object that only computes something on assignment. For complicated expressions this can be a huge win, e.g. r=a+b+c+d will read from a,b,c,d, compute sum of the 4 on the fly, and write into r without temporary copies in memory. However, also makes Eigen’s source code outright scary, and hard to debug or optimize. Anyway, based on the old pics there http://eigen.tuxfamily.org/index.php?title=Benchmark http://eigen.tuxfamily.org/index.php?title=Benchmark they are more or less comparable. Things like alpha·X+beta·Y were much faster in Eigen (probably due to that lazy evaluation thing), Hessenberg was much faster in MKL, in general they are close.
  - gnufx 6y ago
    > When I last checked, MKL was much faster than its competitors That has never been generally true in my experience measuring over the years. It has been true at times for specific cases, e.g. OpenBLAS until it got avx512 support on a par with MKL (at least for serial DGEMM -- I've forgotten quite how the rest of level 3 goes).
adrian_b 6y ago
The new Zen 3 cores are expected to have a higher AVX throughput per cycle than all Intel CPUs, except the most expensive models of Xeon Gold, Platinum or W and the HEDT i9 models that have dual AVX-512 FMA units. The cheaper models with only one AVX-512 FMA unit have a lower throughput, which will be exceeded by Zen 3, even at the same clock frequency. For multi-threaded tasks, Zen 3 CPUs will have a higher clock-frequency than any Intel CPU, so it is expected that any older Intel CPU will be beaten easily. It remains to be seen which will be the performances of the Ice Lake Server CPUs, to be launched before the end of the year. However, miracles are not expected, because these are using the older Intel 10 nm technology, not the improved one used by Tiger Lake.
- gnufx 6y ago
  > The cheaper models with only one AVX-512 FMA unit have a lower throughput, which will be exceeded by Zen 3, even at the same clock frequency. I think there are relatively few with only one FMA (and there's no way of interrogating them at runtime, sigh) but, yes, if you know you have one, you use AVX2 for GEMM kernels, as a specific example. For general computational workloads, you're likely better off with more AVX2 cores and high memory bandwidth, even without whatever improvements there are in Zen 3.
  - pepoluan 6y ago
    > and there's no way of interrogating them at runtime, sigh Well, directly, no. Indirectly ... find the model and use a lookup table to find how many FMA units it has :P
    - gnufx 6y ago
      I've been through the exercise. It's obviously tedious and fragile, and not always possible, as far as I remember. I can't give an example of something that wasn't listed, but I at least have queries against D series and i7/i9.
  - adrian_b 6y ago
    Most or all workstation CPUs with AVX-512 (Xeon W or Core i9) have indeed dual 512-bit FMA units. However most server models with AVX-512 have only one 512-bit FMA unit. That includes all Xeon Bronze, all Xeon Silver and a few Xeon Gold. All Xeon Platinum and most Xeon Gold have dual units, but those are extremely expensive.
gnufx 6y ago
> There are also things like Intel MKL There are also things like OpenBLAS, and BLIS (which AMD support).

by f311a 6y ago

Const-me 6y ago

> is still preferred because of better AVX support AVX1 and AVX2 performance is on par. For instance, vmulpd AVX1 instruction is faster on AMD, 3 versus 4 cycles. vpaddd AVX2 instruction is same at 1 cycle latency. vfmadd132pd FMA instruction is slightly faster on Intel, 4 versus 5 cycles. Throughput is the same across these two. I was looking at AMD Zen2 versus Intel Ice Lake. Some Intel chips have AVX512. Still, many practical applications don’t need that amount of SIMD wideness, and these who do are often a good fit for GPGPUs. > There are also things like Intel MKL There’re vendor-agnostic equivalents like Eigen.

f311a 6y ago
For some computer vision tasks, Tensorflow is much faster when you have AVX512. Also https://www.intel.com/content/www/us/en/artificial-intelligence/posts/tensorflow-optimizations-intel-xeon-scalable-processor.html https://www.intel.com/content/www/us/en/artificial-intellige...
- Const-me 6y ago
  These results were achieved on dual-socket Xeon E5 2699v4 (the architecture is 5 years old and has no AVX512, they optimized for AVX2) and on Xeon Phi 7250 (that thing does have AVX512 but that’s not a processor, a specialized accelerator with 68 cores). Also Tensorflow is awesome fit for GPGPUs and is usually way faster on them.
DoofusOfDeath 6y ago
IIUC, Intel uses the term "AVX-512" as an umbrella term, and different processors support different subsets of "AVX-512" instructions [0]. AFAIK this is a break from previous Intel nomenclature, where any processor supporting e.g. "SSE4.2" instructions was guaranteed to support all SSE4.2 instructions. I'm concerned that sometimes this causes confusion when talking about processor -- software compatibility. [0] https://en.wikipedia.org/wiki/AVX-512 https://en.wikipedia.org/wiki/AVX-512
- loeg 6y ago
  I think the consideration GP was trying to make is that Zen (at least 1 and 2 — and I haven't heard otherwise for 3) do not support 512-bit wide AVX registers at all.
singhrac 6y ago
> There’re vendor-agnostic equivalents like Eigen. That's looking at the wrong layer of the hierarchy, I think. There are many open-source linear algebra libraries, but iirc they all link against something that has a BLAS/LAPACK API. That might be something like MKL, OpenBLAS, ATLAS, etc. When I last checked, MKL was much faster than its competitors, and is only available (at full speed) on Intel CPUs. Has that changed?
- Const-me 6y ago
  > iirc they all link against something that has a BLAS/LAPACK API Eigen can consume these I think, but they are optional. It has it’s own implementation of these, written in manually vectorized C++, with intrinsics, up to and including AVX512 (controlled with macros). For parallelization it uses OpenMP provided by the compiler (also controlled with a macro). > Has that changed? It’s hard to directly compare Eigen to the rest of them. They don’t do the same thing. One feature of Eigen is lazy evaluation. Expressions like a+b or a×b don’t return another matrix or vector; they return a placeholder object that only computes something on assignment. For complicated expressions this can be a huge win, e.g. r=a+b+c+d will read from a,b,c,d, compute sum of the 4 on the fly, and write into r without temporary copies in memory. However, also makes Eigen’s source code outright scary, and hard to debug or optimize. Anyway, based on the old pics there http://eigen.tuxfamily.org/index.php?title=Benchmark http://eigen.tuxfamily.org/index.php?title=Benchmark they are more or less comparable. Things like alpha·X+beta·Y were much faster in Eigen (probably due to that lazy evaluation thing), Hessenberg was much faster in MKL, in general they are close.
- gnufx 6y ago
  > When I last checked, MKL was much faster than its competitors That has never been generally true in my experience measuring over the years. It has been true at times for specific cases, e.g. OpenBLAS until it got avx512 support on a par with MKL (at least for serial DGEMM -- I've forgotten quite how the rest of level 3 goes).

adrian_b 6y ago

The new Zen 3 cores are expected to have a higher AVX throughput per cycle than all Intel CPUs, except the most expensive models of Xeon Gold, Platinum or W and the HEDT i9 models that have dual AVX-512 FMA units. The cheaper models with only one AVX-512 FMA unit have a lower throughput, which will be exceeded by Zen 3, even at the same clock frequency. For multi-threaded tasks, Zen 3 CPUs will have a higher clock-frequency than any Intel CPU, so it is expected that any older Intel CPU will be beaten easily. It remains to be seen which will be the performances of the Ice Lake Server CPUs, to be launched before the end of the year. However, miracles are not expected, because these are using the older Intel 10 nm technology, not the improved one used by Tiger Lake.

gnufx 6y ago
> The cheaper models with only one AVX-512 FMA unit have a lower throughput, which will be exceeded by Zen 3, even at the same clock frequency. I think there are relatively few with only one FMA (and there's no way of interrogating them at runtime, sigh) but, yes, if you know you have one, you use AVX2 for GEMM kernels, as a specific example. For general computational workloads, you're likely better off with more AVX2 cores and high memory bandwidth, even without whatever improvements there are in Zen 3.
- pepoluan 6y ago
  > and there's no way of interrogating them at runtime, sigh Well, directly, no. Indirectly ... find the model and use a lookup table to find how many FMA units it has :P
  - gnufx 6y ago
    I've been through the exercise. It's obviously tedious and fragile, and not always possible, as far as I remember. I can't give an example of something that wasn't listed, but I at least have queries against D series and i7/i9.
- adrian_b 6y ago
  Most or all workstation CPUs with AVX-512 (Xeon W or Core i9) have indeed dual 512-bit FMA units. However most server models with AVX-512 have only one 512-bit FMA unit. That includes all Xeon Bronze, all Xeon Silver and a few Xeon Gold. All Xeon Platinum and most Xeon Gold have dual units, but those are extremely expensive.

gnufx 6y ago

> There are also things like Intel MKL There are also things like OpenBLAS, and BLIS (which AMD support).