3 ms·
Much of what I do is vectorized, so the more fma units and the side wider, the faster. So avx512 > avx2 with two 256 bit units > avx2 with two 128 bit units. B
by celrod 8y ago
Much of what I do is vectorized, so the more fma units and the side wider, the faster. So avx512 > avx2 with two 256 bit units > avx2 with two 128 bit units.
But my work (statistics) seems poorly represented in benchmarks and the community at large. Ryzen is in the later group, and gets rave reviews.
While most press I've seen talking about avx512 has been negative.
Yet I get a lot more flops out of the 10-core i9 7900x than the similarly priced 16-core Threadripper 1950x in simulations.
A big part of that is probably doing most of my work in Julia, where because the code you run is compiled locally the first time a function/argument type combination is called (a) code will be generated for all your CPU's features, and (b) it is easy to encourage aggressive optimizations to vectorized code by the compiler, eg by parameterizing arrays by their bounds.
R benchmarks will not see benefits at all, as much of R's math libraries are terribly unoptimized -- the default BLAS doesn't even use any runtime dispatch! (Thankfully it's easy to change, but the benchmarks all reflect that reality -- and most people's personal experiences too).
Similar likely applies to many NumPy benchmarks, but depending on the distributor of those binaries, the situation likely isn't better. Either generic unoptimized binaries, or shipping with something like Intel MKL which, while the fastest linear algebra library for Intel processors, dispatches to slow code paths for AMD processors -- a very unfair choice for benchmarking. I'm less less familiar with NumPy though, so perhaps there's a booth fair and well optimized distribution.
I fear the future will move against my use cases.
- marmaduke 8y agoIf you want fat vector units you should be on a GPU.
- dragontamer 8y agoYes, in principle. A dedicated GPU will always be "fatter" in SIMD than a CPU. But a CPU has latency benefits. Most noticeably, it takes only a few clock cycles for an Intel CPU to transfer data from its RAX register -> AVX vector registers, compute the solution, and transfer it back to the "scalar world". This level of data-transfer is done within single-digit nanoseconds. In contrast, any GPU memory transfer takes hundreds-of-nanoseconds to microseconds... an order of 100x to 10,000x slower than transferring between Scalar-code and AVX-code on a CPU. ---------- So GPUs are good for bulk compute (big matrix multiplies, Deep Learning, etc. etc.)... but I'm definitely interested in the low-latency uses of SIMD. For example: Consider a SIMD-based Bloom Filters to augment a Hash Table (or any data-structure, really). Multi-hash systems like Cuckoo-hashes have also been successfully implemented with CPU-SIMD acceleration. You can use CPU-accelerated SIMD to accelerate CPU-based algorithms. And CPUs certainly can benefit from fatter, and better designed, SIMD instruction sets.
- Filligree 8y agoUsing AVX instructions clocks down your CPU by a lot, though. It isn't usually worth it unless you're running mostly AVX.
- PeCaN 8y ago> it takes only a few clock cycles for an Intel CPU to transfer data from its RAX register -> AVX vector registers, compute the solution, and transfer it back to the "scalar world". unfortunately, it takes millions of cycles for the CPU to switch to the higher-power state necessary for AVX instructions, so your latency really isn't that much better. the radfft guy did a good writeup https://gist.github.com/rygorous/32bc3ea8301dba09358fd2c64e02d774 https://gist.github.com/rygorous/32bc3ea8301dba09358fd2c64e0...
- celrod 8y agoYes. My code is mostly AVX, so regardless of down-clocking I see substantial performance boosts. But I've found getting into GPU computing extremely difficult. Admittedly, part of that may be having bought an AMD Vega graphics card instead of NVidea (not a fan of vendor locking, and AMD is building an open source stack with ROCm/HIP). My code does lots of different things, many small, but given a billion iterations, it can add up. If you're running a Monte Carlo simulation over millions of different data sets, fitting each iteration with Markov Chain Monte Carlo, with a model that has a few long for loops and requires (automatic) differentiation and some number of special functions... It's not hard to rack up slow execution times out of small pieces. Part of the problem is I need to actually find a GPU project that's accessible for me, so I can gain some experience, and learn what's actually possible. How I could actually organize computation flow. It's easy with a CPU. Break up Monte Carlo simulations among processors. Depending on the model being fit, as well as the method, different computations can vary dramatically in width. But optimizing is as simple as breaking up all those computations into the appropriately sized AVX units (and libraries + compilers auto-vectorization will often handle most of that!), so wider units directly translate to faster performance. Part of the problem is I don't really know how to think about a GPU. Can I think of the Vega GPU as a processor with 64 cores, a SIMD width of 64 (64 * 64 = advertised "4096), and more efficient gather loads/scatter store operations? If there were a CPU like that, compiler + library calls and macros (including libraries you've written or wrapped yourself) go a long way so you can quickly write well optimized code. I really need to dedicate the time to learn more. My 7900x processor is about 4x faster at gemm than my 1950x, but 10x slower than the Vega graphics card that cost less money. I see the potential. To start, I need to find a GPU project that lets me really get experimenting and figure out how programs can even look. Is there an Agner Fog of GPGPU?
- ip26 8y agoPeople aren't benchmarking avx512 because very little is coded to use it yet. You are a small niche. The future will probably include more avx512, but not yet.