3 ms·
> Is it worth the hassle only in rare cases? Vectorization is not always faster. It's important to understand that modern processors can perform work on >100 i
by nickysielicki 7y ago
> Is it worth the hassle only in rare cases?
Vectorization is not always faster. It's important to understand that modern processors can perform work on >100 instructions in a given cycle, and not all instructions take equal amounts of time. So reducing a dozen instructions to a single instruction doesn't necessarily mean that the single instruction is going to be faster.
- atq2119 7y agoThis is misleading, at least if you're looking at a single core. A single core can have >100 instructions in flight, but most of them will be waiting for their operands to arrive. The theoretical maximum throughput is gated by the instruction decoders and execution units, and is an order of magnitude lower.
- nickysielicki 7y agoThat's fair to call me out on that, what I said wasn't clear. I think the broader point still stands though: don't look at the number of ISA instructions and think that fewer instructions necessarily means performance improves. I think that we also miss a piece of the puzzle in terms of what happens once you leave ISA instructions and get into uops. There's magic inside there and it's completely opaque.
- vardump 7y ago> So reducing a dozen instructions to a single instruction doesn't necessarily mean that the single instruction is going to be faster. Are you saying dozen scalar instructions can be faster than one vector instruction? That's wrong 100% of time on modern CPUs.
- nickysielicki 7y agoYeah, I think there exists a case where this is possible. Let me play around with it this weekend and I'll comment here if I find anything.
- vardump 7y agoI know it can happen on some older ARM designs, microcontrollers and weird DSP-like etc. chips. But I can't think of any case on modern x86 chips at least. Some low-performance ARMv7/8 designs can split NEON SIMD instructions into multiple clock cycles, but I think even then NEON is going to perform better.
- nickysielicki 7y agoI started digging into it but I don't have an intel machine with linux easily available and that's a prerequisite for looking at microop performance. I think you win.
- vardump 7y agoI don't need to win, I just want the truth to win. In other words, I'd consider it a win if I, you or anyone else learns something new. SIMD is highly optimized at this point, with sustainable two instructions per clock throughput. It's hard to imagine scalar getting anywhere near even at 4 inst/clk rates.
- jcranmer 7y agoOne corner case that exists is that using AVX instructions imposes a frequency limit, although this isn't the case for SSE instructions. There exist some vector instructions that are going to be slower than a non-equivalent sequence of scalar instructions: VPGATHER is going to be an easy such case. However, I doubt there are going to be any cases where a vector instruction will take fewer clock cycles than its equivalent scalarized instructions. There are some where it might be equivalent--a vector of 2 elements performing an operation that can be issued twice a cycle is an easy example--but I can't think of any where it would be worse. If that were the case, then you should just implement the operation in hardware by scalarizing the uops (and some instructions appear to be so implemented--e.g., gather/scatter).
- vardump 7y agoVPGATHER is actually significantly faster than scalar loads at least on Skylake, possibly on some earlier CPUs as well (Broadwell?). On Haswell where it was first introduced... yeah, not very fast, like you mentioned.