9 ms·
SIMD Everywhere: 0.5.0
- microcolonel 6y agoCareful when invoking (some) AVX-512 instructions, because having a process using them on just one hardware thread can cripple your entire system and hurt overall performance in workloads where the kernel or another process on the system is doing a lot of the work.
- NullPrefix 6y agoHow do cloud vendors handle this on shared machines? Disable AVX?
- w0utert 6y agoNot sure how many cloud vendors are running CPU's with AVX-512 support, but I imagine that if they do you either have to pay for exclusive use of a single CPU (all cores), or thath they indeed disable AVX-512 instructions by patching the cpuid flags? Note that the downclocking is only really a problem with AVX-512 and much less so with AVX2 and (AFAIK) not at all with AVX.
- wtallis 6y agoThe big cloud guys get custom parts with non-standard clock speeds, which may include changes to how AVX512 clocking is handled.
- BeeOnRope 6y agoThe cloud vendors (at least the ones I checked) have SKUs with fairly limited downclocking, but more importantly the license based downclocking is per-core. So as long as you get at least a core to yourself, the noisy neighbor problem is reduced in that you affect your own CPU speed only. For more fine-grained sharing, sure, if the key before you was running AVX-512 you'll get a lower frequency for the initial part of your timeslice, but it should pop back to full speed in under 1 ms.
- lorenzhs 6y agoWhile that's true, I don't see how this is relevant to SIMDe? SIMDe lets you compile code using (e.g.) SSE/AVX intrinsics for ARM targets, using the target platform's SIMD intrinsics when possible. It doesn't even officially include support for AVX512 yet. Anyway, for anyone looking for details on what the OP mentioned, it's an implementation detail of Intel CPUs, see https://en.wikichip.org/wiki/intel/frequency_behavior#Base.2C_Non-AVX_Turbo.2C_and_AVX_Turbo https://en.wikichip.org/wiki/intel/frequency_behavior#Base.2...
- microcolonel 6y ago> it's an implementation detail of Intel CPUs While that may be technically true, since Intel is the only CPU vendor implementing AVX-512, the reason that severe downclocking is warranted is rooted in physics, and may not be resolvable given this ISA design. General vector ISAs like RISC-V "V" may have less trouble, since it is clearer how to design a vector unit not to interfere with scalar access to memory/caches. also, in the article they specifically mention adding shims for several AVX-512 ISAs: > We also have rapidly progressing implementations of many other extensions including NEON, AVX2, SVML, and several AVX-512 extensions (AVX-512F, AVX-512BW, AVX-512VL, etc.). if it's not obvious to you, this means that on machines with AVX-512 hardware will be invoking those instructions directly.
- gameswithgo 6y agoThis is just an easy to understand fact about SIMD so people always post it.
- proverbialbunny 6y agoWhile it's not the most relevant, I don't know a lot about SIMD so I'm grateful to hear it.
- nemequ1729 6y agoIt does include support for AVX-512, it's just still a work in progress. AVX-512 is enormous (IIRC ~ 4k functions), so it will be a while before it is fully supported. If you're targeting AVX-512 but don't have hardware that supports AVX-512 (like all AMD CPUs), SIMDe can be quite nice. The result is much faster than Intel's SDE and it's just native code that you can use your normal debugger on.
- mainland 6y agoIf you're doing new development and not opposed to using C++, I recommend xsimd, which provides a higher-level interface to architecture-specific SIMD instructions: https://github.com/xtensor-stack/xsimd https://github.com/xtensor-stack/xsimd
- lorenzhs 6y agoSIMDe is sort of the reverse: running existing code using SIMD intrinsics of platform A (e.g., Intel SSE) on another platform B (e.g., ARM). It's a great boon for portability of existing SIMD code.
- pjmlp 6y agoWhy not SYCL then? Other than possible available backends.
- nemequ1729 6y agoApart from the fact that you have to rewrite your code, the big disadvantage to something like this is that it's a bit slower. With SIMDe you're still free to use functions like `_mm_maddubs_epi16` and they'll be really fast on x86, but still work everywhere. With xsimd (and similar libraries) you're generally limited to the lowest common denominator. FWIW, if abstraction layers are your thing you might want to look at std::experimental::simd (https://github.com/VcDevel/std-simd https://github.com/VcDevel/std-simd) instead. Google's Highway (https://github.com/google/highway https://github.com/google/highway) is also pretty interesting.
- lloydatkinson 6y agoVery cool. This is effectively a replacement for the now unfortunately abandoned Yeppp library which I used with C# - though modern .NET has SIMD now too. https://news.ycombinator.com/item?id=10232395 https://news.ycombinator.com/item?id=10232395
- stagger87 6y agoThis would not replace yepppp. This is more like a library yeppp would use. If you need to replace yeppp look at something like Intel MKL or Ipps.
- Const-me 6y agoI sometimes write code portable across SSE4 and NEON, and I'm not sure this is going to work fast enough for that. There're important unique features. SSE has shuffles, pack/unpack, movemask, 64-bit doubles, testzero, float rounds, blends, integer averages, float square roots and dot product. NEON has interleaved RAM load/stores, vectors operators with scalar other argument, byte swap, rotate, bit scan and population count, and versions of all instructions processing 8-byte long vectors. That's enough differences that I have to adjust both algorithms and data structures to be portable between them. I'm not convinced it's possible to do automatically.
- dragontamer 6y agoShuffle, pack/unpack, movemask, blends (SSE/AVX) and interleaved load/stores, byte swap (NEON) are "just" data-movement instructions. All of them can be implemented (with obvious slowdowns) with a conditional write to memory, then a conditional read from memory. Yeah, its inefficient to do it like this, but this "write then read" pattern really gives us an idea of what's really going on between the registers in a pack/pshufb/whatever instruction. On AMD GPUs, there's a fully arbitrary crossbar between SIMD-lanes allowing for arbitrary movement. The two instructions are just "permute" and "b-permute" (backwards permute), roughly correlating to gather and scatter respectively. On NVidia GPUs, perm and bperm are both implemented in PTX, but instead read/write to L1 or __shared__ memory. NVidia GPUs likely have a crossbar to L1 memory to make this instruction very fast. --------- The solution is to implement perm and bperm on AVX. Its already half-implemented: pshufb is equivalent to GPU-permute. CPUs are just missing the backwards permute. I'm pretty confident that pack/unpack, blends, interleaved load/stores, and more could all be implemented as pshufb and a hypothetical "backwards pshufb". Version 1.0 could be an NVidia-like "write to L1 cache" sort of implementation too, if full crossbars are too expensive at the hardware layer. ----------- So the question is: how should we write code today? CPUs of today do not implement this feature, but CPUs of the future might. I think specifying the memory-moves explicitly, and then working on a "pshufb compiler/optimizer" of sorts is what we need.
- Const-me 6y ago> the question is: how should we write code today? The only thing that matters today is how fast the code works on today’s hardware. To illustrate, see how I have emulated neon’s vst3q_s16 with SSE: https://github.com/Const-me/DtsDecoder/blob/master/Utils/store3-sse.hpp https://github.com/Const-me/DtsDecoder/blob/master/Utils/sto... However, not all of them can be emulated in a fast enough manner. > specifying the memory-moves explicitly, and then working on a "pshufb compiler/optimizer" of sorts is what we need. I’ll consider that approach when I’ll see that compiler/optimizer working, and producing output comparable to manually-written code. Until then I think that’s a “sufficiently smart compiler” class of problems. These are rarely ever solved, IMO. For example, we don’t have generally useful auto-vectorizers in C compilers despite two decades of R&D, even the best of them (clang, intel) are still very limited.
- deleted 6y ago[deleted]
- gameswithgo 6y agoA similar library in Rust: https://github.com/jackmott/simdeez https://github.com/jackmott/simdeez
- lorenzhs 6y agoThat library is not even remotely similar to SIMDe. The goal of SIMDeez is to provide an abstraction over different SIMD instruction sets (different versions of the x86 SIMD instructions: SSE2, SSE4.1, AVX2). The goal of SIMDe is to let you run code using platform-specific intrinsics on machines that don't have these instructions, e.g. run code with SSE/AVX intrinsics on an ARM-based CPU. They're very different things.
- chuckcode 6y agoSome matrix math libraries like Eigen[1] support vectorization via SSE, AVX, NEON, etc. and also use cache friendly algorithms for larger matrices. Highly recommended when you don't need to go quite as low level as individual instructions. [1] http://eigen.tuxfamily.org http://eigen.tuxfamily.org
- corysama 6y agoIf you need to work on large matrices, Eigen is highly recommended. If you need to do a stream of custom processing in small steps, Eigen is not a good fit. Eigen is a super complicated, template-heavy library that can compile very heavyweight tasks into very efficient code (eventually). For custom work, https://github.com/VcDevel/std-simd https://github.com/VcDevel/std-simd is working it's way into the C++ standard.
- Const-me 6y ago> Eigen is a super complicated, template-heavy library I agree, but that feature allows to apply optimizations by specializing these templates. Works for both micro-optimizations (in their pbroadcast4<__m256d> they do 4 loads, on many CPUs AVX2 can do better with a single load + shuffles) and replacing large parts of Eigen (I was able to improve performance of conjugate gradient solver by moving the sparse matrix into a SIMD-optimized structure).
- jasonzemos 6y agoI'd like to commend the authors for embarking on this. Complex ISA's are an unfortunate reality for performance as advances in cycles-per-second on a single core are negligible. The divergence of these increasingly complex ISA's among platforms weigh heavily on competitive application developers. As someone who is interested in writing cross-platform SIMD code the most valuable asset to me is a library or compiler that can generate the instructions dynamically from otherwise normal'ish looking C/C++. This is the most powerful mode of development in my experience. Clang already does this remarkably. I can write with standard C++ syntax (albeit awkwardly) and maybe using a few custom types with `__attribute__((vector_size(x)))` and not have to involve explicit intrinsics except perhaps for a very small number of leaf operations that cannot be expressed. At this time Clang has the upper-hand on GCC: the latter cannot generate code which scales between platforms utilizing different vector sizes. For example, if you try to perform an operation on a 256-bit vector using a 128-bit target: Clang will seamlessly generate two 128-bit operations; GCC will fall back entirely to scalar. My assumption is that developments in Clang for ARM's SVE have carried over to generating scalable code for other platforms, but nevertheless it is remarkable. I don't believe that writing functions comprised of hand-crafted lists of intrinsics is the best way forward. Undoubtedly it's worked for projects, even quite well to ship stable software -- but it scales and adapts poorly in a fast-developing and diverse market of hardware. For example, years ago I wrote a simple `tolower(string)` implementation using an assemblage of 128-bit standard-Intel SSE2 statements and today the instructions it produces are exactly the same as the day that I wrote it. All I can hope for is that 256-bit capable architectures can execute two of my operations at once. That's not ideal.
- throwaway189262 6y agoIf you just need "big" numbers, Rust supports 128 bit int on all platforms using Clangs code generation
- moonchild 6y agoNo need to reach for rust; just use __(u)int128_t. Works with gcc, too.
- 6y ago