5 ms·

> SIMD assembly can give performance improvements of 10-20x or even higher for many types of real-world code. Using SIMD intrinsics is often preferable. For se

> SIMD assembly can give performance improvements of 10-20x or even higher for many types of real-world code. Using SIMD intrinsics is often preferable. For set-in-stone inner loops, the extra performance you can wring out by hand-writing in assembly may be worth it, especially on compilers with shitty support for intrinsics. For SIMD-based leaf functions that are going to be inlined in many contexts, intrinsics have the advantage since the compiler can schedule and register allocate the instructions piecewise rather than treating them as an indivisible block with hard-wired registers (GCC-style clobber lists are supposed to help the compiler do that with inline assembly, but it sucks in practice). My opinion is that every programmer should learn enough assembly to fluently read the code generated by their compiler, but most of them should not be writing it. Debugging is the number one pay-off, but keeping tabs on what the compiler's optimizer does with your C code is a close second. There was a good illustration of this last week on Hacker News. Someone submitted a StackOverflow thread where they were competitively coding a fast fixed-size sort based on sorting networks. The tentative winner at the time was using some god awful hand-cooked branchless code for min/max that was completely defeating the compiler's attempts at generating the simple and optimal cmp r0, r1; mov r2, r1; cmovg r1, r0; cmovg r0, r2 instruction sequence. That would have instantly obvious if they had the habit of reading their compiler's assembly. The result is that they were leaving a ~3x performance gain on the floor.

deleted 15y ago
[deleted]
DarkShikari 15y ago
Intrinsics tend to be incredibly difficult to read (and write) compared to hand-written assembly. For practical purposes, it's write-only code, and C simply doesn't provide the syntactical niceties necessary to easily write assembly code. Inlining SIMD functions is typically not possible in most applications unless you're compiling a dozen versions, one for each possible CPU, since different CPUs will use different SIMD functions that are optimized for their performance characteristics. Nevermind the fact that intrinsics generally give atrocious performance compared to properly-written code, as your average 3-year-old can probably allocate registers better than gcc.
- psykotic 15y ago
  Intrinsics are about the same level of readability in my experience. Although it's annoying that compilers tend to pile on prefixes to the instruction mnemonics and thereby make everything longer. If that's a big problem for you, just use your own short-hand macro wrappers. > Inlining SIMD functions is typically not possible in most applications unless you're compiling a dozen versions, one for each possible CPU, since different CPUs will use different SIMD functions that are optimized for their performance characteristics. I mainly care about games when it comes to high-performance programming, so this is not an issue for me. Consoles are obviously fixed platforms, and on PC any game is likely to have high enough performance requirements across the board that you can compile your executable against a SSE3 min-spec target without excluding users because of that. > Nevermind the fact that intrinsics generally give atrocious performance, as your average 3-year-old can probably allocate registers better than gcc. I'm glad we agree that GCC is a pile of shit (and not just in this respect). MSVC's performance with intrinsics has also gone up and down with versions but by now it's pretty solid.
  - DarkShikari 15y ago
    I mainly care about games when it comes to high-performance programming, so this is not an issue for me. Consoles are obviously fixed platforms, and on PC any game is likely to have high enough GPU requirements that you can compile your executable against a SSE3 min-spec target without excluding users. The instruction set available is often not the most relevant thing. In many cases, the optimal code will differ wildly between CPUs regardless of the available instruction sets. Let's look at some quick examples of performance characteristics: Athlon 64: Slow SSE2 unit, MMX is often faster than SSE2 for many functions. Phenom: Very fast SSE2 unit, but missing SSSE3 support, so can't take use the same optimized functions in many cases as the Core 2 and above. Significantly higher instruction latency than Core 2, so needs significantly more pipelining. Core 2 Conroe: SSE shuffle operations (punpck, etc) are excruciatingly slow: 4/2 for shuffles that use both arguments as input data (e.g. punpckldq), 2/2 for those that use only one argument (e.g. pshufb). Cacheline-split loads are extraordinarily painful (equivalent to an L1 cache miss, or ~14 cycles). Core 2 Penryn: Same as Conroe, but shuffles are fast now (1/1 for everything basically). Nehalem: Cacheline-split loads are cheap now (2 cycles), and shuffles have doubled throughput compared to Penryn (1/0.5). This doesn't even get into the more instruction-specific messiness, like how some CPUs like movddup and movhlps on integer data while others don't. It's not uncommon to have 3 or 4 assembly functions just to cover the latest CPUs -- not even counting old ones. Fortunately for our sanity, these are usually templated from a single function, with small changes to the relevant areas created via macros.
wtallis 15y ago
Not to rain on your otherwise good comment, but last week's SO question about fixed-size sorts was targeting a GPU, where avoiding branches is actually really important. That ugly branchless code is really bad for a speculative out-of-order complicated core like modern x86, but your simple scalar compare/move code would really suck running on a GPU. That SO question actually did a pretty good job at illustrating your point, because most people assumed that code that works well on their CPU would work well on a GPU, without stopping to think about the architectural differences.
- psykotic 15y ago
  If you read back to my comment in that thread, I explicitly touched on the GPU vs CPU issue. They were benchmarking and ranking algorithms based on CPU performance while allegedly investigating the problem for GPU implementation. Pretty silly of them, but that's besides the point. CUDA and OpenCL and even shader languages like HLSL, GLSL and Cg all have branchless conditional move instructions. They've always had them going back to the days before GPU cores had branching. That's what you should use here, not some homegrown bit-bashing crap. For the higher-level languages like CUDA and OpenCL, the compilers have no problem generating branchless conditional moves from C code that uses branches in a simple and transparent way. The point is that the 'simple and optimal instruction sequence' I mentioned in my top post has a 1:1 equivalent on every modern CPU and GPU.

deleted 15y ago

[deleted]

DarkShikari 15y ago

Intrinsics tend to be incredibly difficult to read (and write) compared to hand-written assembly. For practical purposes, it's write-only code, and C simply doesn't provide the syntactical niceties necessary to easily write assembly code. Inlining SIMD functions is typically not possible in most applications unless you're compiling a dozen versions, one for each possible CPU, since different CPUs will use different SIMD functions that are optimized for their performance characteristics. Nevermind the fact that intrinsics generally give atrocious performance compared to properly-written code, as your average 3-year-old can probably allocate registers better than gcc.

psykotic 15y ago
Intrinsics are about the same level of readability in my experience. Although it's annoying that compilers tend to pile on prefixes to the instruction mnemonics and thereby make everything longer. If that's a big problem for you, just use your own short-hand macro wrappers. > Inlining SIMD functions is typically not possible in most applications unless you're compiling a dozen versions, one for each possible CPU, since different CPUs will use different SIMD functions that are optimized for their performance characteristics. I mainly care about games when it comes to high-performance programming, so this is not an issue for me. Consoles are obviously fixed platforms, and on PC any game is likely to have high enough performance requirements across the board that you can compile your executable against a SSE3 min-spec target without excluding users because of that. > Nevermind the fact that intrinsics generally give atrocious performance, as your average 3-year-old can probably allocate registers better than gcc. I'm glad we agree that GCC is a pile of shit (and not just in this respect). MSVC's performance with intrinsics has also gone up and down with versions but by now it's pretty solid.
- DarkShikari 15y ago
  I mainly care about games when it comes to high-performance programming, so this is not an issue for me. Consoles are obviously fixed platforms, and on PC any game is likely to have high enough GPU requirements that you can compile your executable against a SSE3 min-spec target without excluding users. The instruction set available is often not the most relevant thing. In many cases, the optimal code will differ wildly between CPUs regardless of the available instruction sets. Let's look at some quick examples of performance characteristics: Athlon 64: Slow SSE2 unit, MMX is often faster than SSE2 for many functions. Phenom: Very fast SSE2 unit, but missing SSSE3 support, so can't take use the same optimized functions in many cases as the Core 2 and above. Significantly higher instruction latency than Core 2, so needs significantly more pipelining. Core 2 Conroe: SSE shuffle operations (punpck, etc) are excruciatingly slow: 4/2 for shuffles that use both arguments as input data (e.g. punpckldq), 2/2 for those that use only one argument (e.g. pshufb). Cacheline-split loads are extraordinarily painful (equivalent to an L1 cache miss, or ~14 cycles). Core 2 Penryn: Same as Conroe, but shuffles are fast now (1/1 for everything basically). Nehalem: Cacheline-split loads are cheap now (2 cycles), and shuffles have doubled throughput compared to Penryn (1/0.5). This doesn't even get into the more instruction-specific messiness, like how some CPUs like movddup and movhlps on integer data while others don't. It's not uncommon to have 3 or 4 assembly functions just to cover the latest CPUs -- not even counting old ones. Fortunately for our sanity, these are usually templated from a single function, with small changes to the relevant areas created via macros.

wtallis 15y ago

Not to rain on your otherwise good comment, but last week's SO question about fixed-size sorts was targeting a GPU, where avoiding branches is actually really important. That ugly branchless code is really bad for a speculative out-of-order complicated core like modern x86, but your simple scalar compare/move code would really suck running on a GPU. That SO question actually did a pretty good job at illustrating your point, because most people assumed that code that works well on their CPU would work well on a GPU, without stopping to think about the architectural differences.

psykotic 15y ago
If you read back to my comment in that thread, I explicitly touched on the GPU vs CPU issue. They were benchmarking and ranking algorithms based on CPU performance while allegedly investigating the problem for GPU implementation. Pretty silly of them, but that's besides the point. CUDA and OpenCL and even shader languages like HLSL, GLSL and Cg all have branchless conditional move instructions. They've always had them going back to the days before GPU cores had branching. That's what you should use here, not some homegrown bit-bashing crap. For the higher-level languages like CUDA and OpenCL, the compilers have no problem generating branchless conditional moves from C code that uses branches in a simple and transparent way. The point is that the 'simple and optimal instruction sequence' I mentioned in my top post has a 1:1 equivalent on every modern CPU and GPU.