4 ms·

Anything that's worth spending time to do fast is worth spending time writing SIMD assembly for. You can get 5x, 10x, 20x, or more performance increases just b

Anything that's worth spending time to do fast is worth spending time writing SIMD assembly for. You can get 5x, 10x, 20x, or more performance increases just by using the vector instructions given to you by the CPU. Until a magic compiler appears that can make proper use of them (read: never), hand-coded assembly will be critical for almost any application for which performance is critical, especially multimedia processing.

gruseom 17y ago
A compiler can't be made to make proper use of vector instructions? Why is that?
- angelbob 17y ago
  In many cases it can, but then you have to be sure that it keeps doing so. You can write test cases for that, but in many cases it's easier to verify by just using the instructions directly. If you don't write the compiler, you have to make assumptions about when and how it can/will use those instructions, and often you assume wrong, particularly across compiler upgrades.
- DarkShikari 17y ago
  It is an extraordinarily difficult problem to transform scalar code into vector instructions. The only way to get even passable output from a vectorizing compiler is to write the code as vectors to begin with, such as with cross-platform assembly tools like Orc. And even then you'll often end up significantly worse off than if you wrote the assembly by hand. A run of Intel's compiler on the C versions of our DSP functions resulted in a grand total of one vectorization, which was done terribly, too.
  - jrockway 17y ago
    The problem is that you used C, which doesn't have any syntax to represent meta-information about the problem you're trying to solve. When you write out C code to, say, add a list of numbers, it's hard for the compiler to optimize that. But it's very easy for the compiler when you tell it "sum this list of numbers".
- JoachimSchipper 17y ago
  As I understand - and I may be mistaken - the problem is that it's very hard to make sure that the emitted code is correct in the presence of raw pointers à la C. I am told that Fortran does better than C here; there is a reason it is still widely used in the scientific computing community, after all. This is also part of the reasoning behind C99's new restrict keyword.
ntoshev 17y ago
Well, you can still use C with GCC and Intel intrinsics I guess.
- DarkShikari 17y ago
  Intrinsics are still worse (performance-wise) than hand-coded assembly, for two reasons. The first reason is the whole category of optimizations that the compiler is worse than a human at (like register allocation) or cannot do effectively at all (messing with calling conventions, computed jumps). The second reason is more subtle: in any case that you abstract yourself from some part of a problem, you inherently create a less efficient solution. For example, intrinsics mean that you don't have to manually allocate registers. But this also means that if your algorithm uses too many registers and it would be more efficient to modify it to require fewer (and thus not need spills), you will have no way of knowing such a thing. By insulating yourself from that layer of complexity, you've also limited your ability to make higher-level optimizations that improve lower-level performance. This applies on practically every level possible: any method of abstraction, no matter how well designed, will always in some fashion reduce the maximum performance you can achieve. Of course, this doesn't mean abstraction is bad--it provides an often-useful tradeoff between developer time and performance.
  - ntoshev 17y ago
    I agree with your arguments, but they apply to non-vector code as well. Perhaps higher level programming using the right primitives is good enough in most cases.

gruseom 17y ago

A compiler can't be made to make proper use of vector instructions? Why is that?

angelbob 17y ago
In many cases it can, but then you have to be sure that it keeps doing so. You can write test cases for that, but in many cases it's easier to verify by just using the instructions directly. If you don't write the compiler, you have to make assumptions about when and how it can/will use those instructions, and often you assume wrong, particularly across compiler upgrades.
DarkShikari 17y ago
It is an extraordinarily difficult problem to transform scalar code into vector instructions. The only way to get even passable output from a vectorizing compiler is to write the code as vectors to begin with, such as with cross-platform assembly tools like Orc. And even then you'll often end up significantly worse off than if you wrote the assembly by hand. A run of Intel's compiler on the C versions of our DSP functions resulted in a grand total of one vectorization, which was done terribly, too.
- jrockway 17y ago
  The problem is that you used C, which doesn't have any syntax to represent meta-information about the problem you're trying to solve. When you write out C code to, say, add a list of numbers, it's hard for the compiler to optimize that. But it's very easy for the compiler when you tell it "sum this list of numbers".
JoachimSchipper 17y ago
As I understand - and I may be mistaken - the problem is that it's very hard to make sure that the emitted code is correct in the presence of raw pointers à la C. I am told that Fortran does better than C here; there is a reason it is still widely used in the scientific computing community, after all. This is also part of the reasoning behind C99's new restrict keyword.

ntoshev 17y ago

Well, you can still use C with GCC and Intel intrinsics I guess.

DarkShikari 17y ago
Intrinsics are still worse (performance-wise) than hand-coded assembly, for two reasons. The first reason is the whole category of optimizations that the compiler is worse than a human at (like register allocation) or cannot do effectively at all (messing with calling conventions, computed jumps). The second reason is more subtle: in any case that you abstract yourself from some part of a problem, you inherently create a less efficient solution. For example, intrinsics mean that you don't have to manually allocate registers. But this also means that if your algorithm uses too many registers and it would be more efficient to modify it to require fewer (and thus not need spills), you will have no way of knowing such a thing. By insulating yourself from that layer of complexity, you've also limited your ability to make higher-level optimizations that improve lower-level performance. This applies on practically every level possible: any method of abstraction, no matter how well designed, will always in some fashion reduce the maximum performance you can achieve. Of course, this doesn't mean abstraction is bad--it provides an often-useful tradeoff between developer time and performance.
- ntoshev 17y ago
  I agree with your arguments, but they apply to non-vector code as well. Perhaps higher level programming using the right primitives is good enough in most cases.