5 ms·

> A Programmer’s Perspective I'm a programmer too but I don't share that perspective. In my line of work, AVX512 is useless. This won't change until at least

> A Programmer’s Perspective I'm a programmer too but I don't share that perspective. In my line of work, AVX512 is useless. This won't change until at least 25% market penetration on clients, which may or may not happen. Not all programmers are writing server code. Also, very small count of them are Intel partners with early access to the chips. > In AVX terminology, intra-lane movement is called a `shuffle` while inter-lane movement is called a `permute` I don't believe that's true. _mm_shuffle_ps( x, x, imm ) and _mm_permute_ps( x, imm ) are 100% equivalent. Also, _mm256_permute_ps permutes within lanes, _mm256_permutevar8x32_ps permutes across lanes.

AdamProut 5y ago
If your building software as a service that changes this calculus a fair bit. You only need to wait for AWS/Azure/GCP/CSP of choice to support the hardware (you don't need to wait for mass market use of it). For example, many (all?) database as a service offerings now days run on very precisely configured hardware.
- Const-me 5y ago
  > many (all?) database as a service offerings now days run on very precisely configured hardware I'm not sure AVX512 is a win even in that case. AMD Epyc peaks at 64 cores/socket, Intel Xeon at 40 cores/socket. It's not immediately obvious AMD's performance advantage over Intel is smaller than the Intel-only win from AVX512.
  - dragontamer 5y ago
    Are sockets the best measurement? AMD has a pretty complex network inside that socket: 8 dies + a switch. The Xeon 40 core is just one die, which means L3 cache and memory is more unified. L3 cache is probably king (data may not fit, but maybe some indexes?), but it's not obvious if EPYCs separate L3 caches are comparable to Xeons (or Power)
    - Const-me 5y ago
      AMD is much faster overall. This page has a benchmark of a MySQL database, the slide is called MariaDB: https://www.servethehome.com/amd-epyc-7763-review-top-for-this-generation/2/ https://www.servethehome.com/amd-epyc-7763-review-top-for-th... The only benchmark where AMD didn't have substantial advantage is chess, I think the reason is AMD's slow pdep/pext instructions.
      - dragontamer 5y ago
        The 6258R is just 28 cores though, 56 total after dual socket. AMD Zen3 has single cycle pext / pdep now and the chess benchmarks are better as a result.
        Const-me 5y ago
        Good point. Indeed, for DB-only workloads the Xeon appears to be better: https://www.phoronix.com/scan.php?page=article&item=intel-xeon-8380-linux&num=6 https://www.phoronix.com/scan.php?page=article&item=intel-xe... Still, this doesn’t mean Epyc is only good for HPC, e.g. this is some enterprise Java-based benchmark where it’s substantially faster: https://www.anandtech.com/show/16594/intel-3rd-gen-xeon-scalable-review/9 https://www.anandtech.com/show/16594/intel-3rd-gen-xeon-scal... BTW, on my job I don’t do databases, but I do quite a lot of HPC stuff.
        dragontamer 5y ago
        Add to the fact that MariaDB doesn't have AVX512 instructions used yet, and you can see that Intel's unified L3 cache is just better for database-like workloads than AMD's split L3 cache. x265 is more of an AVX512 test scenario, which the Xeon 40-core also is demonstrating proficiency at over and above the EPYC 64 core. ------------ I'm mostly impressed at how well the split "chiplet" strategy is doing in all the other benchmarks however. The AMD "I/O die" is clearly a winner. Its probably not a latency winner, but it is efficiently giving the memory bandwidth and distributing it to all of the dies.
janwas 5y ago
> In my line of work, AVX512 is useless. This won't change until at least 25% market penetration on clients, which may or may not happen. Why not use it where available? github.com/google/highway lets you write code once using 'platform-independent intrinsics', and generate AVX2/AVX-512/NEON/SVE into a single binary; dynamic dispatch runs the best available codepath for the current CPU. Disclosure: I am the main author. Happy to discuss.
- Const-me 5y ago
  > Why not use it where available? AFAIK things like that (highway, OpenMP 4.0+, automatic vectorizers) are only good for vertical operations. When the only thing you’re doing is billions of vertical operations, computing that thing on CPU is often a poor choice because GPUs are way faster at that, and way more power efficient too. Therefore, when I optimize CPU-running things, that code is usually not a good fit for GPUs. Sometimes the data size is too small (PCIe latency gonna eat all the profit from GPU), sometimes too many branches, but most typical reason is horizontal operations. An example is multiplying 6x24 by 24x1 matrices. A good approach is a loop with 12 iterations, in the loop body 2x _mm256_broadcast_sd, then _mm256_blend_pd (these 3 instructions are making 3 vectors [v1,v1,v1,v1], [v1,v1,v2,v2], [v2,v2,v2,v2] ), then 3x _mm256_fmadd_pd with memory source operands. Then after the loop add 12 values into 6 with shuffles and vector adds.
  - janwas 5y ago
    > things like that (highway, OpenMP 4.0+, automatic vectorizers) are only good for vertical operations. I agree autovec and #pragma omp simd struggle with any kind of shuffles, but Highway does not belong in that category - it offers more control, similar to intrinsics. There are about two dozen non-vertical ops (see https://github.com/google/highway/blob/master/g3doc/quick_reference.md#combine https://github.com/google/highway/blob/master/g3doc/quick_re... and "blockwise" and "swizzle") and it sounds like you want Set and ConcatUpperLower. In your experience, have you needed any non-vertical ops that are not in that list?
    - Const-me 5y ago

AdamProut 5y ago

If your building software as a service that changes this calculus a fair bit. You only need to wait for AWS/Azure/GCP/CSP of choice to support the hardware (you don't need to wait for mass market use of it). For example, many (all?) database as a service offerings now days run on very precisely configured hardware.

Const-me 5y ago
> many (all?) database as a service offerings now days run on very precisely configured hardware I'm not sure AVX512 is a win even in that case. AMD Epyc peaks at 64 cores/socket, Intel Xeon at 40 cores/socket. It's not immediately obvious AMD's performance advantage over Intel is smaller than the Intel-only win from AVX512.
- dragontamer 5y ago
  Are sockets the best measurement? AMD has a pretty complex network inside that socket: 8 dies + a switch. The Xeon 40 core is just one die, which means L3 cache and memory is more unified. L3 cache is probably king (data may not fit, but maybe some indexes?), but it's not obvious if EPYCs separate L3 caches are comparable to Xeons (or Power)
  - Const-me 5y ago
    AMD is much faster overall. This page has a benchmark of a MySQL database, the slide is called MariaDB: https://www.servethehome.com/amd-epyc-7763-review-top-for-this-generation/2/ https://www.servethehome.com/amd-epyc-7763-review-top-for-th... The only benchmark where AMD didn't have substantial advantage is chess, I think the reason is AMD's slow pdep/pext instructions.
    - dragontamer 5y ago
      The 6258R is just 28 cores though, 56 total after dual socket. AMD Zen3 has single cycle pext / pdep now and the chess benchmarks are better as a result.
      - Const-me 5y ago
        Good point. Indeed, for DB-only workloads the Xeon appears to be better: https://www.phoronix.com/scan.php?page=article&item=intel-xeon-8380-linux&num=6 https://www.phoronix.com/scan.php?page=article&item=intel-xe... Still, this doesn’t mean Epyc is only good for HPC, e.g. this is some enterprise Java-based benchmark where it’s substantially faster: https://www.anandtech.com/show/16594/intel-3rd-gen-xeon-scalable-review/9 https://www.anandtech.com/show/16594/intel-3rd-gen-xeon-scal... BTW, on my job I don’t do databases, but I do quite a lot of HPC stuff.
        dragontamer 5y ago
        Add to the fact that MariaDB doesn't have AVX512 instructions used yet, and you can see that Intel's unified L3 cache is just better for database-like workloads than AMD's split L3 cache. x265 is more of an AVX512 test scenario, which the Xeon 40-core also is demonstrating proficiency at over and above the EPYC 64 core. ------------ I'm mostly impressed at how well the split "chiplet" strategy is doing in all the other benchmarks however. The AMD "I/O die" is clearly a winner. Its probably not a latency winner, but it is efficiently giving the memory bandwidth and distributing it to all of the dies.

janwas 5y ago

> In my line of work, AVX512 is useless. This won't change until at least 25% market penetration on clients, which may or may not happen. Why not use it where available? github.com/google/highway lets you write code once using 'platform-independent intrinsics', and generate AVX2/AVX-512/NEON/SVE into a single binary; dynamic dispatch runs the best available codepath for the current CPU. Disclosure: I am the main author. Happy to discuss.

Const-me 5y ago
> Why not use it where available? AFAIK things like that (highway, OpenMP 4.0+, automatic vectorizers) are only good for vertical operations. When the only thing you’re doing is billions of vertical operations, computing that thing on CPU is often a poor choice because GPUs are way faster at that, and way more power efficient too. Therefore, when I optimize CPU-running things, that code is usually not a good fit for GPUs. Sometimes the data size is too small (PCIe latency gonna eat all the profit from GPU), sometimes too many branches, but most typical reason is horizontal operations. An example is multiplying 6x24 by 24x1 matrices. A good approach is a loop with 12 iterations, in the loop body 2x _mm256_broadcast_sd, then _mm256_blend_pd (these 3 instructions are making 3 vectors [v1,v1,v1,v1], [v1,v1,v2,v2], [v2,v2,v2,v2] ), then 3x _mm256_fmadd_pd with memory source operands. Then after the loop add 12 values into 6 with shuffles and vector adds.
- janwas 5y ago
  > things like that (highway, OpenMP 4.0+, automatic vectorizers) are only good for vertical operations. I agree autovec and #pragma omp simd struggle with any kind of shuffles, but Highway does not belong in that category - it offers more control, similar to intrinsics. There are about two dozen non-vertical ops (see https://github.com/google/highway/blob/master/g3doc/quick_reference.md#combine https://github.com/google/highway/blob/master/g3doc/quick_re... and "blockwise" and "swizzle") and it sounds like you want Set and ConcatUpperLower. In your experience, have you needed any non-vertical ops that are not in that list?
  - Const-me 5y ago