6 ms·

This is a good decision. SIMD.js never made sense to me. Writing vector code that outperforms scalar code requires not only fine-grained control over issues lik

This is a good decision. SIMD.js never made sense to me. Writing vector code that outperforms scalar code requires not only fine-grained control over issues like alignment, but also requires targeting specific SIMD instruction sets. You must write your vector code with the emitted assembly in mind, because if you fall off the fast path and the compiler switches to scalar code or does a library call or something, you've lost any performance gain and then some. I think this highlights the absurdity of writing vector code in higher-level constructs rather than the direct ASM. Writing in ASM is not that hard, and the gains can be enormous. I've seen graphics operations sped up by 5x or more, just by writing hand-optimized vector ops in assembly. This is one of Visual Studio's core strengths vs GCC. Writing an __asm { } block is much more pleasant than the syntax GCC forces on you. Unfortunately this inertia is one reason games tend to be windows-only, even in the post-Apple era. In other words, phrases like "You must write your vector code with the emitted assembly in mind, because if you fall off the fast path and the compiler switches to scalar code" invokes a feeling of "What? Why does the compiler get any say in what the vector code looks like? Oh, right, everybody shies away from writing ASM by hand nowadays." Which is valid, but sometimes having ultra control is extremely nice. It just feels cool to get a 2x speedup by being persistent and clever. (Those speedups come with future technical debt, though, so there are good arguments against it.)

sddfd 9y ago
The compiler should vectorize functions automatically, without user intervention. Writing assembly code to use SIMD instructions is a vote of no confidence on your compiler. If your compiler of choice can't do automatic vectorization switch to one that can, or get people to implement that feature in your compiler of choice.
- dagss 9y ago
  Well, the compilers don't do a good enough job, end of story (at least using C; not sure about OpenCL). To achieve top performance it is necessary to make sure everything in the computation is organized in the right way. And try out multiple strategies on the high level of the program, to see what achieves best performance at the low level. At that point, trying to obfuscate your SIMD logic in regular C so that the C compiler maybe figure out what you meant is just frustrating anyway. What you are saying is basically "you don't need top performance, just deal with what the compiler gives you". But sometimes you work on stuff where the whole point is to be best in class in performance (libsharp in my case). If you think that what you are suggesting is possible, feel free to improve the compilers so that normal C code beats OpenBLAS. You will be famous. I will wait....
  - sddfd 9y ago
    Writing inline assembly to get SIMD performance is likely to cost immense time of an architecture expert and doesn't scale. So my point stands: If a compiler can't produce vectorised code, the compiler needs to be improved. Spending time on improving the compiler is sustainably spent time. Spending time on programming SIMD in assembly by hand likely is not. I believe you that the vectorization support at the moment is not good enough, and I completely understand that not everyone can spend time on improving the compiler. However, it seems you know exactly what a compiler should be able to do and what feedback from the compiler, or annotations for the compiler, would be helpful for your use-cases where optimization somehow didn't figure out what to vectorize. I was just pointing out that communicating with compiler people and getting them to improve the compiler is likely a better move than asking for better support for inline assembly.
    - dagss 9y ago
      The problem is not communication. To quote Paul Graham in a keynote from some years back: People have been waiting for "sufficiently smart compilers" to cut programmers out of the loop for 30 years. It is a neat idea in theory. In practice it is just very hard. You need to design the algorithm for the hardware, so it is sort of an AI problem. Programmers do fill a role usually in coming up with the best algorithms... The point is simply that C doesn't have the concepts you want to program with. Vectorization only gets you so far, there are many other things you can do with AVX/SSE. It doesn't need to be ASM, it could be supersets of C instead (like CUDA). Intrinsic functions for AVX is what I use and I don't see the problem with using those. It sort of is such a superset of C. In practice a few numerical computations (BLAS, FFTs, etc) are reused a lot by many. It is worth it to write those few libraries in assembly (or at least intrinsics). For the rest we just have to live with a small performance penalty. I am just saying "if you want to go the extra mile to get high performance, you can beat compilers". In most scenarios of course it makes most sense to not bother.
      - sddfd 9y ago
        I get your point. However, I don't think it is useful to consider a compiler to be a replacement for programmers. Compilers are tools for programmers. The more time a compiler saves a programmer, the better it is. I think there is a middle ground between writing inline assembly and fully automatic vectorization that would be less time intensive than manual vectorization and more predictable and available earlier than fully automatic vectorization. I wonder what would have to be done to find it and provide support for it in GCC/LLVM.
- throwaway2048 9y ago
  Compilers should just be sufficiently smart.
- brian-armstrong 9y ago
  This isn't a realistic comment whatsoever. Compilers are nowhere near what you're describing
slededit 9y ago
it was nicer until support for __asm was removed in the 64-bit compiler. Now if you want assembly you need to write the whole function in it.
zvrba 9y ago
> Writing an __asm { } block is much more pleasant than the syntax GCC forces on you. Or just use intrinsics and let the compiler arrange register allocation, scheduling, etc. for you.
- YZF 9y ago
  In my experience the compiler does a pretty poor job at this. If you really care about every cycle you want to hand code these... At least this was the case ~6 years ago, haven't looked at more recent compilers.
  - pcwalton 9y ago
    Register allocation is a solved problem. The optimal solution can be found in polynomial time for SSA code. SIMD is no different.
    - acqq 9y ago
      Solved in the papers or in the available products? Can you please write a little more?
    - DSrcl 9y ago
      Register allocation in general is np-complete[1][2]. Having polynomial-time coloring algorithm for SSA doesn't make the allocation optimal -- you still need to "lower" the SSA to proper machine code. [1] http://www.cs.cmu.edu/afs/cs/academic/class/15745-s07/www/papers/pereira-fossacs06.pdf http://www.cs.cmu.edu/afs/cs/academic/class/15745-s07/www/pa... [2] http://pharm.ece.wisc.edu/wddd/2006/papers/wddd_05.pdf http://pharm.ece.wisc.edu/wddd/2006/papers/wddd_05.pdf
  - zvrba 9y ago
    I've recently worked on optimizing a piece of integer-based AVX2 code which was running in a tight loop (dot product of 128-dimensional vectors with 8-bit components). I inspected the assembly generated by MSVC17, and it did a really good job; no extra loads or stores or register spills. It was "as if" hand-written. > In my experience the compiler does a pretty poor job at this. A year ago I've worked with a guy optimizing ffmpeg's PRORES encoder; he also used AVX2 intrinsics. The result? 45% speed improvement (gcc) w/o any of the fuss of hand-coded assembly. [1] Sadly, the ffmpeg team suffers from the same prejudices as you, and won't accept intrinsics-based code, even though their "policy" is to accept all patches giving improvement of just a couple of percent points. EDIT: Ironically, they'd most probably accept the output of gcc -S og forfeit future optimizations implemented in gcc and other compilers. It is as if experienced programmers are forever stuck with their past experiences and won't ever look back. [1] https://ffmpeg.org/pipermail/ffmpeg-devel/2016-May/194575.html https://ffmpeg.org/pipermail/ffmpeg-devel/2016-May/194575.ht...
vanderZwan 9y ago
> (Those speedups come with future technical debt, though, so there are good arguments against it.) Have you seen Halide? Might be a way to have your cake and eat it too. http://halide-lang.org/ http://halide-lang.org/
asveikau 9y ago
Last I knew __asm doesn't work on amd64 or arm. Still the case?
- planteen 9y ago
  For what toolchain? GCC supports inline assembly for every architecture I have used it on (x86, x86-64, PPC, SPARC, ARM)
  - asveikau 9y ago
    __asm{} as described in the comment I am replying to is an MSVC extension which lets you do assembly sprinkled with a mix of C identifiers. GCC's equivalent feature (asm() or __asm___()) is quite different, with the list of inputs, outputs, clobbers etc.
deleted 9y ago
[deleted]

sddfd 9y ago

The compiler should vectorize functions automatically, without user intervention. Writing assembly code to use SIMD instructions is a vote of no confidence on your compiler. If your compiler of choice can't do automatic vectorization switch to one that can, or get people to implement that feature in your compiler of choice.

dagss 9y ago
Well, the compilers don't do a good enough job, end of story (at least using C; not sure about OpenCL). To achieve top performance it is necessary to make sure everything in the computation is organized in the right way. And try out multiple strategies on the high level of the program, to see what achieves best performance at the low level. At that point, trying to obfuscate your SIMD logic in regular C so that the C compiler maybe figure out what you meant is just frustrating anyway. What you are saying is basically "you don't need top performance, just deal with what the compiler gives you". But sometimes you work on stuff where the whole point is to be best in class in performance (libsharp in my case). If you think that what you are suggesting is possible, feel free to improve the compilers so that normal C code beats OpenBLAS. You will be famous. I will wait....
- sddfd 9y ago
  Writing inline assembly to get SIMD performance is likely to cost immense time of an architecture expert and doesn't scale. So my point stands: If a compiler can't produce vectorised code, the compiler needs to be improved. Spending time on improving the compiler is sustainably spent time. Spending time on programming SIMD in assembly by hand likely is not. I believe you that the vectorization support at the moment is not good enough, and I completely understand that not everyone can spend time on improving the compiler. However, it seems you know exactly what a compiler should be able to do and what feedback from the compiler, or annotations for the compiler, would be helpful for your use-cases where optimization somehow didn't figure out what to vectorize. I was just pointing out that communicating with compiler people and getting them to improve the compiler is likely a better move than asking for better support for inline assembly.
  - dagss 9y ago
    The problem is not communication. To quote Paul Graham in a keynote from some years back: People have been waiting for "sufficiently smart compilers" to cut programmers out of the loop for 30 years. It is a neat idea in theory. In practice it is just very hard. You need to design the algorithm for the hardware, so it is sort of an AI problem. Programmers do fill a role usually in coming up with the best algorithms... The point is simply that C doesn't have the concepts you want to program with. Vectorization only gets you so far, there are many other things you can do with AVX/SSE. It doesn't need to be ASM, it could be supersets of C instead (like CUDA). Intrinsic functions for AVX is what I use and I don't see the problem with using those. It sort of is such a superset of C. In practice a few numerical computations (BLAS, FFTs, etc) are reused a lot by many. It is worth it to write those few libraries in assembly (or at least intrinsics). For the rest we just have to live with a small performance penalty. I am just saying "if you want to go the extra mile to get high performance, you can beat compilers". In most scenarios of course it makes most sense to not bother.
    - sddfd 9y ago
      I get your point. However, I don't think it is useful to consider a compiler to be a replacement for programmers. Compilers are tools for programmers. The more time a compiler saves a programmer, the better it is. I think there is a middle ground between writing inline assembly and fully automatic vectorization that would be less time intensive than manual vectorization and more predictable and available earlier than fully automatic vectorization. I wonder what would have to be done to find it and provide support for it in GCC/LLVM.
throwaway2048 9y ago
Compilers should just be sufficiently smart.
brian-armstrong 9y ago
This isn't a realistic comment whatsoever. Compilers are nowhere near what you're describing

slededit 9y ago

it was nicer until support for __asm was removed in the 64-bit compiler. Now if you want assembly you need to write the whole function in it.

zvrba 9y ago

> Writing an __asm { } block is much more pleasant than the syntax GCC forces on you. Or just use intrinsics and let the compiler arrange register allocation, scheduling, etc. for you.

YZF 9y ago
In my experience the compiler does a pretty poor job at this. If you really care about every cycle you want to hand code these... At least this was the case ~6 years ago, haven't looked at more recent compilers.
- pcwalton 9y ago
  Register allocation is a solved problem. The optimal solution can be found in polynomial time for SSA code. SIMD is no different.
  - acqq 9y ago
    Solved in the papers or in the available products? Can you please write a little more?
  - DSrcl 9y ago
    Register allocation in general is np-complete[1][2]. Having polynomial-time coloring algorithm for SSA doesn't make the allocation optimal -- you still need to "lower" the SSA to proper machine code. [1] http://www.cs.cmu.edu/afs/cs/academic/class/15745-s07/www/papers/pereira-fossacs06.pdf http://www.cs.cmu.edu/afs/cs/academic/class/15745-s07/www/pa... [2] http://pharm.ece.wisc.edu/wddd/2006/papers/wddd_05.pdf http://pharm.ece.wisc.edu/wddd/2006/papers/wddd_05.pdf
- zvrba 9y ago
  I've recently worked on optimizing a piece of integer-based AVX2 code which was running in a tight loop (dot product of 128-dimensional vectors with 8-bit components). I inspected the assembly generated by MSVC17, and it did a really good job; no extra loads or stores or register spills. It was "as if" hand-written. > In my experience the compiler does a pretty poor job at this. A year ago I've worked with a guy optimizing ffmpeg's PRORES encoder; he also used AVX2 intrinsics. The result? 45% speed improvement (gcc) w/o any of the fuss of hand-coded assembly. [1] Sadly, the ffmpeg team suffers from the same prejudices as you, and won't accept intrinsics-based code, even though their "policy" is to accept all patches giving improvement of just a couple of percent points. EDIT: Ironically, they'd most probably accept the output of gcc -S og forfeit future optimizations implemented in gcc and other compilers. It is as if experienced programmers are forever stuck with their past experiences and won't ever look back. [1] https://ffmpeg.org/pipermail/ffmpeg-devel/2016-May/194575.html https://ffmpeg.org/pipermail/ffmpeg-devel/2016-May/194575.ht...

vanderZwan 9y ago

> (Those speedups come with future technical debt, though, so there are good arguments against it.) Have you seen Halide? Might be a way to have your cake and eat it too. http://halide-lang.org/ http://halide-lang.org/

asveikau 9y ago

Last I knew __asm doesn't work on amd64 or arm. Still the case?

planteen 9y ago
For what toolchain? GCC supports inline assembly for every architecture I have used it on (x86, x86-64, PPC, SPARC, ARM)
- asveikau 9y ago
  __asm{} as described in the comment I am replying to is an MSVC extension which lets you do assembly sprinkled with a mix of C identifiers. GCC's equivalent feature (asm() or __asm___()) is quite different, with the list of inputs, outputs, clobbers etc.

deleted 9y ago

[deleted]