5 ms·
Auto-vectorization is easier to get into than other SIMD-frameworks like CUDA, OpenCL, ROCm, Intel's ISPC and whatnot. But in my experience, auto-vectorizers ar
by dragontamer 3y ago
Auto-vectorization is easier to get into than other SIMD-frameworks like CUDA, OpenCL, ROCm, Intel's ISPC and whatnot. But in my experience, auto-vectorizers are just not as flexible as the proper SIMD-tools.
I'd say auto-vectorization should still be learned by modern high-performance programmers, because its very low-hanging fruit. You barely have to do anything and suddenly your for-loops are AVX512 optimized, though maybe not to the fullest extent possible.
Still, I suggest that programmers also learn how to properly make SIMD code. Maybe intrinsics are too hard in practice, but ISPC, CUDA, and other SIMD-programming environments make things far easier to learn than you might expect.
------------
ISPC in particular is Intel's SIMD programming language, much akin to CUDA except it compiles into AVX512. So for AVX512-like code execution environments, using the ISPC language/compiler is exceptionally useful.
Its harder to learn a new language than to learn a few compiler-options to enable auto-vectorization however. So in practice, auto-vectorization will continue to be used. But for tasks that specifically would benefit from SIMD-thinking, the dedicated ISPC language should be beneficial.
- sva_ 3y ago> I'd say auto-vectorization should still be learned by modern high-performance programmers, because its very low-hanging fruit. Somewhat related: 55 GiB/s FizzBuzz https://codegolf.stackexchange.com/questions/215216/high-throughput-fizz-buzz/236630#236630 https://codegolf.stackexchange.com/questions/215216/high-thr... (discussion: https://news.ycombinator.com/item?id=29031488 https://news.ycombinator.com/item?id=29031488)
- saagarjha 3y agoDefinitely not autovectorized.
- jackmott42 3y agoWhat do you mean by learning auto vectorization? Do you mean learning how to structure your code so that your compiler of choice has a good chance of pulling it off?
- Veliladon 3y agoPretty much. If one wants to sum two arrays instead of for looping through the elements of two arrays one might instead use iterate by chunks so that the compiler can easily tie them all together as a single operation which can then easily vectorize.
- pasmafaute 3y agoIf I recall, you can absolutely loop through the elements in a tight loop and the compiler (e.g. GCC) will auto-vectorize for you (if you have the relevant optimization flag set). The trick with coding for auto-vectorization is to keep your loops small and free of clutter. I don't have the documentation handy but I think you only need to follow a couple rules: - loop must have a defined size (for-loop instead of while-loop) - don't muck with pointers inside the loop (simple pointer increment is okay) - don't modify other variables (only the array should be modified)
- dragontamer 3y agoThe linked document describes Intel's autovectorizer, it's warnings and compiler flags that point out which loops autovectorized or not, as well as listing specific reason codes why. Microsoft, GCC and Clang all do this too, though with different compiler flags and messages. I'd say that the whole point of this document listed here is to build up the programmer to understanding these error messages and specifically know how to fix the errors that causes a autovectorization-fail.
- galangalalgol 3y agoIn rust I've noticed using iterators instead of for loops tends to trigger auto vectorization more often.
- Veliladon 3y agoIn Rust using iterators is almost always better than for loops. The compiler is virtually guaranteed to optimize out bounds checks vs indexing in.
- iamcreasy 3y agoWhat would happen if I try to run a program contains AVX instruction on a system that does not have it? Do I need to recompile the program?
- stevefan1999 3y agosegfault with illegal opcode
- iamcreasy 3y agoSo when I am downloading binary - i.e. Krita/GIMP or any open source project - how do the dev ensure that the common binary will utilize these newer instructions without hitting 'illegal opcode' error?
- fragmede 3y agoDepends on the binary. Compiling to the lowest common denominator is one strategy. The other is to to have the code detect what's available and only use that subset.
- iamcreasy 3y agoInteresting. How do the devs decide on the lowest common denominator? Do you happen to know any blog or resources that explains the process?
- Const-me 3y agoI touched that question in "Hardware Support" section, on page 6 of that article http://const.me/articles/simd/simd.pdf http://const.me/articles/simd/simd.pdf TLDR: SSE1 and SSE2 are parts of the base AMD64 ISA, they are always available while compiling for 64 bits. SSE3, SSE4.1 are old, market penetration is very close to 100%, always available in practice. For the newer extensions like AVX and FMA, the optimal tradeoff depends on the product and the market.
- Genghis_Khan 3y ago> auto-vectorizers are just not as flexible as the proper SIMD-tools What is a "proper" SIMD tool?
- dragontamer 3y agoThere is no such thing as 'failure to autovectorize' in CUDA or OpenCL. All code is vectorized in these languages. SIMD is fundamental to the language model. As such, it's easier to write high performance code in practice. Intel's ISPC is the closest tool that replicates this effect for AVX512.
- mgaunard 3y agoEven more important than learning how to vectorize is learning the basics of computer architecture: register, caches, pipelining, branch prediction, and superscalar execution. One of the common interview questions I ask is computing the sum of an array of doubles. Barely 1% of applicants can even get the basics correctly such as having multiple partial sums. Once you do the work such that your code actually allows execution to happen in parallel, then leveraging superscalar and simd happens automatically.
- listenallyall 3y agoIs asking a question that fewer than 1% of applicants "can even get the basics correct" -- which implies that only a tiny fraction of 1% actually provide a fully complete and correct answer -- a good use of either the applicants' or the interviewer's time? Is it effective in distinguishing candidates, or are you only seeking 1 success (fewer, actually) per 100?
- mgaunard 3y ago1% success rate is typical when interviewing developers. Also they get asked multiple exercises, not doing great at one is not eliminatory. Being able to quickly identify whether an individual is exceptionally good is very important to ensure the employer can fast-track them and be responsive quickly enough that he doesn't accept another offer.
- preseinger 3y agocpu cache line optimization is relevant to a superminority of programming contexts if you're working in those contexts then it's all fine, but it's important to acknowledge that most software development is bottlenecked by concerns far above this level of concern
- mgaunard 3y ago[flagged]
- 3y ago