3 ms·

> Well, of course intrinsics perform just as well as inline assembly when you're doing a single matrix multiply or a single matrix inverse. You won't see any be

by exDM69 13y ago

> Well, of course intrinsics perform just as well as inline assembly when you're doing a single matrix multiply or a single matrix inverse. You won't see any benefit unless you're doing high-throughput computation like thousands of consecutive multiplies. That's when the clever programmer will wipe the floor with your clever compiler. This was exactly what I was doing - high throughput inner loop code with lots of matrix multiplies, etc. I wrote the primitive operations (multiply, inverse, etc) with intrinsics but my high level code was readable and maintainable C code. The compiler-emitted assembly code was at least as good as I could have written by hand, pretty close to the speed of light (ie. pretty close to CPU peak flops and/or memory bandwidth, whichever was the relevant bottleneck). It takes an exceptional programmer and a lot of time to "wipe the floor with the compiler" with hand written assembly code. The compiler's ability to come up with near-optimal instruction scheduling and the ability to quickly change register allocation is pretty hard to match - you can do it if you spend enough time with the Intel manuals but that's not time well spent. By analyzing your compiler output and fine-tuning your intrinsics code, you should be able to get pretty close to the peak performance with less time spent than hand writing the asm code. I stand by my point - a clever compiler and a clever programmer together can get a better job done than either of those individually. This, however, requires a programmer with assembly skills. It's still valuable to be able to write and especially read assembly code.

sillysaurus3 13y ago
I wrote the primitive operations (multiply, inverse, etc) with intrinsics but my high level code was readable and maintainable C code. What I'm saying is that you have to write an algorithm in assembly which has an inner loop. You can't have the body of your inner loop in assembly, but the looping mechanism in C. This is because you can pull out parts of the algorithm from the inside of the loop to outside of it, or you can prevent the compiler from constantly re-loading from memory into SSE registers inside the inner loop, etc. In other words, what I think you're saying is: for (...) { /* inline assembly or intrinsics */ } What I'm saying is: /* start inline assembly preload anything that can be preloaded into SSE registers loop: achieve parts of the overall goal write any finished operations back to main memory prefetch N cachelines ahead end inline assembly */ It may seem like a small difference, but in the former case, you're still doing a "single matrix multiply" (you're doing them one at a time). I think that might be why you're not seeing much of a difference between intrinsics vs inline assembly. Matrix multiply isn't such a good example... there are better cases like texture lookups in a software rasterizer. Also, the prefetching step is vital. It may be you're not seeing performance gains due to lack of prefetching, which is orthogonal to the debate of SSE intrinsics vs SSE inline assembly. Something like VTune can help reveal precisely why your program is going slow. There are many pitfalls, and it's easy to accidentally optimize for the wrong thing and then trick yourself into believing that particular optimization never helps in any circumstance.
- exDM69 13y ago
  No you didn't understand what I was doing nor why I was getting good results. My code was something like: __builtin_prefetch(first cache line); for(a lot of 3d objects) { __builtin_prefetch(next cache line); vec4 q = read_quaternion(), p = read_position(); mat4 m = matrix_product(translate(p), quat_to_mat(q)); mat4 n = inverse_transpose(m); mat4 p = matrix_product(camera_matrix, m); // lots of more matrix-vector-quaternion math here stream_store(m); stream_store(n); .... } In other words, my "high level" code was readable C code, only the primitives (matrix_product, etc) were intrinsics code. What the compiler did is inlined all the primitive ops, store all the values in registers at all times (no loads or stores or register spilling in the inner loop) and finally re-organize the instructions to get near optimal scheduling. In some simpler programs, I got even more benefit from the compiler doing some loop unrolling to keep loads/stores balanced with ALU ops to get very effective latency hiding. Writing that whole loop with assembler would have yielded next to no improvement but that would have sacrificed readability, maintainability and portability. > You can't have the body of your inner loop in assembly, but the looping mechanism in C. You're right in that you can't mix inline assembler and C and get the compiler to optimize it correctly, but using intrinsics you can get the best of both worlds. This is pretty much exactly what I had, except that my "inner loop" primitives were C code+intrinsics that looked like assembly code. Because they were not assembler code, the compiler was able to optimize the whole loop and the end result is something similar to what you show as an example of good performing asm code. If you do end up writing assembler, you have to be sure that it is worth sacrificing the compiler optimizations that would take place otherwise.
  - sillysaurus3 13y ago
    And when the compiler wants to use more SSE registers than are available, it will register spill. A clever programmer may be able to avoid that, and that's where the performance boost comes from. You've convinced me to leave that as an ultra-last-resort though, and just try out intrinsics first. Dumb compilers are probably less of an issue nowadays than in ye olden days of ~5 years ago. Thank you for the thorough explanation!

by exDM69 13y ago

sillysaurus3 13y ago

I wrote the primitive operations (multiply, inverse, etc) with intrinsics but my high level code was readable and maintainable C code. What I'm saying is that you have to write an algorithm in assembly which has an inner loop. You can't have the body of your inner loop in assembly, but the looping mechanism in C. This is because you can pull out parts of the algorithm from the inside of the loop to outside of it, or you can prevent the compiler from constantly re-loading from memory into SSE registers inside the inner loop, etc. In other words, what I think you're saying is: for (...) { /* inline assembly or intrinsics */ } What I'm saying is: /* start inline assembly preload anything that can be preloaded into SSE registers loop: achieve parts of the overall goal write any finished operations back to main memory prefetch N cachelines ahead end inline assembly */ It may seem like a small difference, but in the former case, you're still doing a "single matrix multiply" (you're doing them one at a time). I think that might be why you're not seeing much of a difference between intrinsics vs inline assembly. Matrix multiply isn't such a good example... there are better cases like texture lookups in a software rasterizer. Also, the prefetching step is vital. It may be you're not seeing performance gains due to lack of prefetching, which is orthogonal to the debate of SSE intrinsics vs SSE inline assembly. Something like VTune can help reveal precisely why your program is going slow. There are many pitfalls, and it's easy to accidentally optimize for the wrong thing and then trick yourself into believing that particular optimization never helps in any circumstance.

exDM69 13y ago
No you didn't understand what I was doing nor why I was getting good results. My code was something like: __builtin_prefetch(first cache line); for(a lot of 3d objects) { __builtin_prefetch(next cache line); vec4 q = read_quaternion(), p = read_position(); mat4 m = matrix_product(translate(p), quat_to_mat(q)); mat4 n = inverse_transpose(m); mat4 p = matrix_product(camera_matrix, m); // lots of more matrix-vector-quaternion math here stream_store(m); stream_store(n); .... } In other words, my "high level" code was readable C code, only the primitives (matrix_product, etc) were intrinsics code. What the compiler did is inlined all the primitive ops, store all the values in registers at all times (no loads or stores or register spilling in the inner loop) and finally re-organize the instructions to get near optimal scheduling. In some simpler programs, I got even more benefit from the compiler doing some loop unrolling to keep loads/stores balanced with ALU ops to get very effective latency hiding. Writing that whole loop with assembler would have yielded next to no improvement but that would have sacrificed readability, maintainability and portability. > You can't have the body of your inner loop in assembly, but the looping mechanism in C. You're right in that you can't mix inline assembler and C and get the compiler to optimize it correctly, but using intrinsics you can get the best of both worlds. This is pretty much exactly what I had, except that my "inner loop" primitives were C code+intrinsics that looked like assembly code. Because they were not assembler code, the compiler was able to optimize the whole loop and the end result is something similar to what you show as an example of good performing asm code. If you do end up writing assembler, you have to be sure that it is worth sacrificing the compiler optimizations that would take place otherwise.
- sillysaurus3 13y ago
  And when the compiler wants to use more SSE registers than are available, it will register spill. A clever programmer may be able to avoid that, and that's where the performance boost comes from. You've convinced me to leave that as an ultra-last-resort though, and just try out intrinsics first. Dumb compilers are probably less of an issue nowadays than in ye olden days of ~5 years ago. Thank you for the thorough explanation!