3 ms·

> Writing SSE assembly (not intrinsics) by hand can sometimes give you over a 10x speedup in an inner loop of a computationally expensive problem, like CPU bone

by exDM69 13y ago

> Writing SSE assembly (not intrinsics) by hand can sometimes give you over a 10x speedup in an inner loop of a computationally expensive problem, like CPU bone animation of vertices. My experiences from assembly code vs. intrinsics are just the opposite from yours. What I got out of some intrinsics heavy C code from GCC and Clang was way better than I could ever write. Secondly, using inlineable functions, I could get interprocedural optimization from the compiler. I only needed to write the low level primitives (4x4 matrix multiply, matrix inverse, etc) with intrinsics and the high level code would be inlined and optimized. Adding assembly code to the mix will inhibit compiler optimization. Additionally, the code I wrote could be retargeted for ARM NEON just by switching compilers. I was using mostly the compilers' vector extensions instead of CPU-specific intrinsics (ie. __builtin_shuffle, not __mm_shuffle_ps). > And while it's convenient to use intrinsics to generate SSE (intrinsics look like ordinary C functions and save you from having to write actual assembly), it's impossible for them to always be as efficient as a clever programmer. I'm pretty sure that generating perfectly efficient SSE from SSE intrinsics is an NP-complete problem. So a clever programmer will always be able to boost performance more than a compiler. Looking at the assembly code generated by GCC and Clang, I can see that the compiler does near optimal instruction scheduling and register allocation. On the other hand, the compiler was rather poor in doing instruction selection. I needed to pay a little attention to the code I wrote to avoid GCC spilling registers to the stack, Clang was a bit better. So in my experience, a clever programmer writes poorer assembly code than a clever programmer using a clever compiler and intrinsics. You might need to look at the emitted code from the compiler and do some fine adjustments, so assembly skills are still needed. I bet there are still cases where you can beat the compiler in an isolated problem, but when it comes to maintainability, re-targeting and the amount of effort needed to write the code, writing assembly code isn't worth it most of the time.

sillysaurus3 13y ago
Well, of course intrinsics perform just as well as inline assembly when you're doing a single matrix multiply or a single matrix inverse. You won't see any benefit unless you're doing high-throughput computation like thousands of consecutive multiplies. That's when the clever programmer will wipe the floor with your clever compiler. (Well, probably only by a factor of 2x. But still, 2x is better than not-2x in the exceptionally rare case when performance is critical.) I wasn't arguing that people should generally do this. The tradeoffs are rarely worth it. What I was saying is that humans can be more capable than compiler magic, and when it matters, it can really matter. Sometimes by up to 600x, as the other commenter demonstrated.
- exDM69 13y ago
  > Well, of course intrinsics perform just as well as inline assembly when you're doing a single matrix multiply or a single matrix inverse. You won't see any benefit unless you're doing high-throughput computation like thousands of consecutive multiplies. That's when the clever programmer will wipe the floor with your clever compiler. This was exactly what I was doing - high throughput inner loop code with lots of matrix multiplies, etc. I wrote the primitive operations (multiply, inverse, etc) with intrinsics but my high level code was readable and maintainable C code. The compiler-emitted assembly code was at least as good as I could have written by hand, pretty close to the speed of light (ie. pretty close to CPU peak flops and/or memory bandwidth, whichever was the relevant bottleneck). It takes an exceptional programmer and a lot of time to "wipe the floor with the compiler" with hand written assembly code. The compiler's ability to come up with near-optimal instruction scheduling and the ability to quickly change register allocation is pretty hard to match - you can do it if you spend enough time with the Intel manuals but that's not time well spent. By analyzing your compiler output and fine-tuning your intrinsics code, you should be able to get pretty close to the peak performance with less time spent than hand writing the asm code. I stand by my point - a clever compiler and a clever programmer together can get a better job done than either of those individually. This, however, requires a programmer with assembly skills. It's still valuable to be able to write and especially read assembly code.
  - sillysaurus3 13y ago
    I wrote the primitive operations (multiply, inverse, etc) with intrinsics but my high level code was readable and maintainable C code. What I'm saying is that you have to write an algorithm in assembly which has an inner loop. You can't have the body of your inner loop in assembly, but the looping mechanism in C. This is because you can pull out parts of the algorithm from the inside of the loop to outside of it, or you can prevent the compiler from constantly re-loading from memory into SSE registers inside the inner loop, etc. In other words, what I think you're saying is: for (...) { /* inline assembly or intrinsics */ } What I'm saying is: /* start inline assembly preload anything that can be preloaded into SSE registers loop: achieve parts of the overall goal write any finished operations back to main memory prefetch N cachelines ahead end inline assembly */ It may seem like a small difference, but in the former case, you're still doing a "single matrix multiply" (you're doing them one at a time). I think that might be why you're not seeing much of a difference between intrinsics vs inline assembly. Matrix multiply isn't such a good example... there are better cases like texture lookups in a software rasterizer. Also, the prefetching step is vital. It may be you're not seeing performance gains due to lack of prefetching, which is orthogonal to the debate of SSE intrinsics vs SSE inline assembly. Something like VTune can help reveal precisely why your program is going slow. There are many pitfalls, and it's easy to accidentally optimize for the wrong thing and then trick yourself into believing that particular optimization never helps in any circumstance.

by exDM69 13y ago

sillysaurus3 13y ago

Well, of course intrinsics perform just as well as inline assembly when you're doing a single matrix multiply or a single matrix inverse. You won't see any benefit unless you're doing high-throughput computation like thousands of consecutive multiplies. That's when the clever programmer will wipe the floor with your clever compiler. (Well, probably only by a factor of 2x. But still, 2x is better than not-2x in the exceptionally rare case when performance is critical.) I wasn't arguing that people should generally do this. The tradeoffs are rarely worth it. What I was saying is that humans can be more capable than compiler magic, and when it matters, it can really matter. Sometimes by up to 600x, as the other commenter demonstrated.

exDM69 13y ago
> Well, of course intrinsics perform just as well as inline assembly when you're doing a single matrix multiply or a single matrix inverse. You won't see any benefit unless you're doing high-throughput computation like thousands of consecutive multiplies. That's when the clever programmer will wipe the floor with your clever compiler. This was exactly what I was doing - high throughput inner loop code with lots of matrix multiplies, etc. I wrote the primitive operations (multiply, inverse, etc) with intrinsics but my high level code was readable and maintainable C code. The compiler-emitted assembly code was at least as good as I could have written by hand, pretty close to the speed of light (ie. pretty close to CPU peak flops and/or memory bandwidth, whichever was the relevant bottleneck). It takes an exceptional programmer and a lot of time to "wipe the floor with the compiler" with hand written assembly code. The compiler's ability to come up with near-optimal instruction scheduling and the ability to quickly change register allocation is pretty hard to match - you can do it if you spend enough time with the Intel manuals but that's not time well spent. By analyzing your compiler output and fine-tuning your intrinsics code, you should be able to get pretty close to the peak performance with less time spent than hand writing the asm code. I stand by my point - a clever compiler and a clever programmer together can get a better job done than either of those individually. This, however, requires a programmer with assembly skills. It's still valuable to be able to write and especially read assembly code.
- sillysaurus3 13y ago
  I wrote the primitive operations (multiply, inverse, etc) with intrinsics but my high level code was readable and maintainable C code. What I'm saying is that you have to write an algorithm in assembly which has an inner loop. You can't have the body of your inner loop in assembly, but the looping mechanism in C. This is because you can pull out parts of the algorithm from the inside of the loop to outside of it, or you can prevent the compiler from constantly re-loading from memory into SSE registers inside the inner loop, etc. In other words, what I think you're saying is: for (...) { /* inline assembly or intrinsics */ } What I'm saying is: /* start inline assembly preload anything that can be preloaded into SSE registers loop: achieve parts of the overall goal write any finished operations back to main memory prefetch N cachelines ahead end inline assembly */ It may seem like a small difference, but in the former case, you're still doing a "single matrix multiply" (you're doing them one at a time). I think that might be why you're not seeing much of a difference between intrinsics vs inline assembly. Matrix multiply isn't such a good example... there are better cases like texture lookups in a software rasterizer. Also, the prefetching step is vital. It may be you're not seeing performance gains due to lack of prefetching, which is orthogonal to the debate of SSE intrinsics vs SSE inline assembly. Something like VTune can help reveal precisely why your program is going slow. There are many pitfalls, and it's easy to accidentally optimize for the wrong thing and then trick yourself into believing that particular optimization never helps in any circumstance.