14 ms·
Is sorted using SIMD instructions
- zakk 9y agoIt would be interesting to see how the SIMD versions work for small arrays. I suspect in this case the naive version better, and this could be the reading why compilers do not convert the code to SIMD instructions...
- gameswithgo 9y agoSIMD tends to always win for pretty small arrays, like around ~25 elements, and probably any size that is evenly divisible by the vector width.
- alfanick 9y agoI always miss multithreaded benchmarks when using SSE/AVX instructions. AFAIK AVX processing units are oversubscribed, there are less of them than CPU cores. I can imagine that running AVX is_sorted (or any other AVX procedure) in multiple threads would be actually slower than running non-vectorized procedure. Of course, that's my purely anecdotal opinion.
- frankchn 9y agoI think AVX units on Intel cores have always been separate (i.e. not shared). AMD Bulldozer processors have a shared floating point unit and some early pipeline stages like the instruction decoder per pair of cores. AMD Zen processors have since reverted to a more conventional design.
- jnordwick 9y agoThat doesnt make sense to me. On haswell/skylake ports 0 and 1 do most of the vector lifting. I don't think cores share any of the vector hardware. Or are you trying to make some claim about hyperthreading?
- alfanick 9y agoI simply may be wrong :) There is other AVX related thing [0] - CPU is underclocked in Turbo when AVX is used. [0]: https://www.anandtech.com/show/11544/intel-skylake-ep-vs-amd-epyc-7000-cpu-battle-of-the-decade/8 https://www.anandtech.com/show/11544/intel-skylake-ep-vs-amd...
- deleted 9y ago[deleted]
- nvartolomei 9y agoHere is an experience report on how AVX-512 instructions impact CPU performance https://blog.cloudflare.com/on-the-dangers-of-intels-frequency-scaling/ https://blog.cloudflare.com/on-the-dangers-of-intels-frequen...
- paulmd 9y agoNote that Skylake-SP and Xeon-W/i7/i9s behave very differently in this regard. On Skylake-SP (eg Xeon Silvers like they're using) it's over 50% clockrate reduction when AVX-512 is in the pipe, on Xeon-W and the HEDT chips it's more like 10-20%. https://twitter.com/InstLatX64/status/934093081514831872 https://twitter.com/InstLatX64/status/934093081514831872
- floatboth 9y agoOn HEDT (and mainstream desktop) you can actually adjust AVX offset manually. With 0 offset and 5GHz clock, you can consume 500W (in Prime95 AVX) :D
- vardump 9y ago> AFAIK AVX processing units are oversubscribed, there are less of them than CPU cores. Typically I see about 2 SIMD instruction throughput per cycle per core on Intel CPUs. SIMD execution units are not shared between cores in any way. Clock throttling might happen, but SIMD is usually still a pretty huge net win.
- nimos 9y agoI'm pretty sure each core has its own AVX units for the version they support. AFAIK the only difference between AVX performance (ignoring clock speed) is gold/platinum(5000+) Xeons have 2x512 FMA ports available but everything else only supports FMA on port 1/2 and not 5. Stabbing a bit in the dark here, it's been a bit since I was looking at this stuff.
- DSingularity 9y agoThat’s only correct for avx3 instructions -1 I.e. 512 bit wide vector unit.
- eloff 9y agoIt's not that the execution units are shared, but that the frequency is throttled when AVX2, AVX512 instructions are encountered. In general AVX512 is not yet worthwhile when overall system throughput is at stake, and you don't have very vector heavy workloads. AVX2 is worthwhile most of the time. As of Haswell one lane of the AVX2 units was powered down when not in use and instructions execute more slowly when first encountering them. It executes them basically by stitching together two SEE operations. But this doesn't necessarily make it slower than SSE, just that the performance benefits might not materialize if there isn't enough AVX2 code being executed. I don't know if Skylake works that way as well.
- pubby 9y agoUnfortunately SSE doesn't speed-up huge data much. It's only fast when everything's in the cache.
- dosshell 9y agoDo you have any explanation of why you think so? And how much is hugh data for you? I work with image processing, >1GiB/s. We optimize for each platform for hand, many embedded platforms but also x86_64. Most of our work is detectors and loss-less compression and we could not do these things in realtime if it weren't for SIMD/MIMD.
- pubby 9y agoI was not very specific. What I meant to get at was that if your bottleneck is memory, then optimizing for instructions is not going to help. is_sort is an example algorithm that's far more dependent on memory than it is on instruction throughput. If your bottleneck isn't memory, then yeah, SIMD is a real boon.
- emerged 9y agoIt's rare to have a meaningful algorithm where memory is the absolute only bottleneck. Not much else aside from memcpy and even that can see a benefit from SIMD tuning on many systems. Especially since high performance software is glad to have even a 1% boost in speed.
- tmandry 9y agoIt’s not all about compute. SIMD instructions allow you to load more bits per instruction cycle. When you are doing sequential access on a modern processor, this can make a huge difference. It allows you to use the memory bandwidth to its full capacity.
- 0xfaded 9y agoI coded up a 5x5 Gaussian blur using NEON instructions. I used the standard seperatable filter technique, one horizontal pass and then one vertical pass. Benchmarking effectively measured 2x the memory round trip time on the raspberry pi. My second implementation rotated 8x16 image blocks using register permute instructions. Only the block borders needed to be cached, meaning everything shared between blocks for a 640x480 image fit in L1. Despite being substantially more complex, I cut the execution time by 30%. Lesson learned, sometimes the easiest performance gains are found by not being naive about memory access. The extra instructions were inconsequential.
- stkdump 9y agoIt seems that the early exit (the return false) in the middle of the loop prohibits the compiler from vectorizing the loop. The compiler can't know if part of the memory is inaccessible and so reading memory not being sure that the loop would run up to this point is illegal. If you introduce a result variable and set it during the loop, the compiler can vectorize the loop. At least icc does, but I didn't play with the compiler settings of the other compilers too much. Also compilers can still exit the loop early and at least MSVC does, because a segfault is UB and can be "optimized away".
- CodeArtisan 9y agoWhen compiling with GCC, the option `-fopt-info-vec-all` gives you information about the vectorization of the code. In this case, GCC reports // the for block <source>:10:24: note: ===== analyze_loop_nest ===== <source>:10:24: note: === vect_analyze_loop_form === <source>:10:24: note: not vectorized: control flow in loop. <source>:10:24: note: bad loop form. <source>:5:6: note: vectorized 0 loops in function. // the if block inside the for block <source>:11:9: note: got vectype for stmt: _4 = *_3; const vector(16) int <source>:11:9: note: got vectype for stmt: _8 = *_7; const vector(16) int <source>:11:9: note: === vect_analyze_data_ref_accesses === <source>:11:9: note: not vectorized: no grouped stores in basic block. <source>:11:9: note: ===vect_slp_analyze_bb=== <source>:11:9: note: ===vect_slp_analyze_bb=== <source>:11:9: note: === vect_analyze_data_refs === <source>:11:9: note: not vectorized: not enough data-refs in basic block. edit: with intel compiler using `-qopt-report=5 -qopt-report-phase=vec -qopt-report-file=stdout` Begin optimization report for: is_sorted(const int32_t *, size_t) Report from: Vector optimizations [vec] LOOP BEGIN at <source>(12,5) remark #15324: loop was not vectorized: unsigned types for induction variable and/or for lower/upper iteration bounds make loop uncountable LOOP END
- konschubert 9y agoAm I reading this right: GCC cannot vectorise because the loop terminates early in case of a non-sorted pair. It thereby contains control flow. I guess that's kind of logical. Even if the compiler recognised the early termination of the loop as the optimisation it is, it would still have to make the decision to give up on it in favour of vectorisation (?)
- justincormack 9y agoEven if you adjust it to not terminate early it finds other reasons (doesnt vectorise boolean operations).
- CJefferson 9y agoActually, this optimisation is illegal. I could line up memory such that it is illegal to read past the first location where the array is not sorted. The original code would be fine, the vectorised code would segfault. Similar annoying problems arise when people try to be clever with C strings, and read past the null. You can write optimisations which work, but they require care.
- danbruc 9y agoi += 7; Wouldn't this cause a sizable performance hit due to being misaligned most of the time?
- secure 9y agoNo: many (most?) modern SIMD instructions don’t require alignment. From the Intel Intrinsics Guide (can’t figure out how to link directly to it, sorry) on _mm_loadu_si128: > Load 128-bits of integer data from memory into dst. mem_addr does not need to be aligned on any particular boundary.
- detaro 9y agoDoesn't need, but is there a performance difference? I seem to remember there is no difference between _mm_load_si128 and _mm_loadu_si128 on modern CPUs, but I'm not sure.
- deleted 9y ago[deleted]
- mattkrause 9y agoloadu is not the best example because its sole purpose is loading unaligned data. There is a separate load for aligned data.
- wmu 9y agoAuthor here. This was true several generations ago (core2, for instance), now the performance penalty is negligible.
- redcalx 9y agoI think the reduction in the number of branches inside the loop is a significant factor here, i.e. the if statement is performed per 4,8 or 16 elements instead of per value element. Branches invoke branch prediction logic which is non trivial, thus even though it may not slow execution it may increase power consumption. On this basis another way of approaching the problem is to loop over the whole array and return the result at the end, but this of course would take N/2 loops on average, whereas the nested if can exit early. A compromise might be to loop over short sub-spans of the array, and do an exit early test at the end of each sub-span. A good sub-span length for scalar code might be around 16, for one because we hit the law of diminishing returns for longer spans. Also I think is_sorted_asc() or is_sorted_ascending() might be a better name if this were for a function in a general purpose library.
- Veedrac 9y agoFWIW HeroicKatora gives an improved version on Reddit: https://www.reddit.com/r/cpp/comments/8bkaj3/is_sorted_using_simd_instructions/dx7jj8u/ https://www.reddit.com/r/cpp/comments/8bkaj3/is_sorted_using...
- lokopodium 9y agoGeneric code yields better results than SSE/AVX optimized ones. I wonder why that could be.
- eloff 9y agoIt's just replacing extra loads with a bunch of other instructions. In reality loads are cheap (and cached), it turns out to be cheaper than doing permutes to shuffle the vectors around.
- foxhill 9y agousing a bit more arcane (but not crazy) method for determining sorted-ness - https://godbolt.org/g/MKN9HP https://godbolt.org/g/MKN9HP - clang, and icc seem to have no problem vectorising the inner loop. gcc manages it too, but emits a huge amount of code, though. and setting -Os seems to stop it from vectorising the code. shame. edit: replaced with more correct version..
- en4bz 9y agoAnother possible implementation which is log^2(n) https://en.wikipedia.org/wiki/Bitonic_sorter https://en.wikipedia.org/wiki/Bitonic_sorter
- monochromatic 9y agoNo, you can’t even read the whole array in O(log^2(n)). It’s not possible to do better than O(n) without “cheating.”
- IAmLiterallyAB 9y agoI think the trick is you can run it in parallel. Which under ideal circumstances may give that kind of performance. But this is the first I've heard of this algorithm.
- nwmcsween 9y agoThis assumes unaligned access is cheap.
- lucb1e 9y agoSee this thread (posted 7 hours before your comment): https://news.ycombinator.com/item?id=16842012 https://news.ycombinator.com/item?id=16842012
- nwmcsween 9y agoWhich is OK in this case but for any of the arch independent (SWAR) bit hacks it would probably be better to have a loop to align.