5 ms·

HN loves SIMD, and there is a "how I hand crafted a SIMD optimization" post doing numbers on here regularly. They're fun posts, and it absolutely speaks to the

HN loves SIMD, and there is a "how I hand crafted a SIMD optimization" post doing numbers on here regularly. They're fun posts, and it absolutely speaks to the fact that writing code that optimizing compilers can robustly and comprehensively turn into good SIMD branches is somewhat of a black art. Which is why you, generally, shouldn't be doing either. You shouldn't rely upon the compiler to figure out your intentions, and you shouldn't be writing SIMD instructions directly unless you're writing a SIMD library or an optimizing compiler. Instead you should reach for one of the many available libraries that not only force you into appropriately structuring your data and calls for SIMD goodness, they're massively more portable and powerful. Google's Highway, for instance, will let you use their abstracted SIMD functions and it provides the optimization whether your target is SSE2-4, AVX, AVX2, AVX512, AVX10, or if you build for ARM NEON or SVE, for any conceivable vector size, or WASM's weird SIMD functions, or RISC-V's RVV, and several more, and when new widths and new options come out, the library adds the support and you might not have to change your code at all. There are loads of libraries like this (xsimd, EVE, SIMDe, etc). They all force you into thinking about structuring your code in a manner that is SIMDable -- instead of hoping the optimizing compiler will figure it out on its own -- and provide targeting for a vast trove of SIMD options without hand-writing for every option. I was going to quickly rewrite the example in Highway just to demonstrate but the Perlin stuff seems to be missing or significantly restructured. "But that is obvious and I'm mad that you commented this" - no, it isn't obvious whatsoever, and this "I hand-rolled some SSE now my app is super awesome look at the microbenchmark results on a very narrow, specific machine" content appears on here regularly, betraying a pretty big influence of beginners who don't know that it's almost certainly the wrong approach.

63 1y ago
This is a valuable viewpoint that lines up somewhat with some other discussion I've seen on the topic [0]. I'd like to see more posts about structuring code for the auto vectorizor (with libraries or otherwise) rather than writing simd by hand. Do you have any documentation you'd recommend? [0] https://matklad.github.io/2023/04/09/can-you-trust-a-compiler-to-optimize-your-code.html https://matklad.github.io/2023/04/09/can-you-trust-a-compile...
jesse__ 1y ago
I disagree pretty strongly with most of what you said, but I'd be very interested in seeing a Highway example and looking at the differences. Take a look through the comments, I left a link to the test bench I made, which contains all the code.
- janwas 1y ago
  Highway author here :) I'm curious what you disagree with, because it all sounds very sensible to me?
  - jesse__ 1y ago
    There's a lot to discuss. First off, a number of statements are nonsense. Take, for example > you shouldn't be writing SIMD instructions directly unless you're writing a SIMD library or an optimizing compiler. Why would writing an optimizing compiler qualify as territory for directly writing SIMD code, but anything else is off the table? That makes no sense at all. Furthermore, I was writing a library. It's just embedded in my game engine. > Instead you should reach for one of the many available libraries This blanket statement is only true in a narrow set of circumstances. In my mind, it requires that you ship on multiple architectures and probably multiple compilers. If you have narrower constraints, it's extremely easy to write your own wrappers (like I did) and not take a dependency. A good trade IMO. Furthermore, someone's got to write the libraries, so doing it yourself as a learning exercise has value. > There are loads of libraries like this [...] and provide targeting for a vast trove of SIMD options without hand-writing for every option. The original commentor seems to be under the impression that using a SIMD library would somehow have produced a better result. The fact is, the library code is super fucking boring. I barely mentioned it in the article because it's basically just boilerplate an LLM could probably spit out, first try. The interesting part of the series is the observation that you can precompute a matrix of intermediates and look them up, instead of recomputing them in the hot loop, effectively trading memory bandwidth for less instructions. A good trade for this algorithm, which saturates the instruction pipelines. The thing the original commentor does get right is the notion that thinking about data layout is important. But, that has nothing to do with the library you're using .. you just have to do it. They seem to be conflating the use of a library with the act of writing wide code, as if you can't do one without the other, which is obviously false. > I was going to quickly rewrite the example in Highway .. Right. I'll believe this when I see it. I could pick it apart more, but.. I think you get my drift.

63 1y ago

This is a valuable viewpoint that lines up somewhat with some other discussion I've seen on the topic [0]. I'd like to see more posts about structuring code for the auto vectorizor (with libraries or otherwise) rather than writing simd by hand. Do you have any documentation you'd recommend? [0] https://matklad.github.io/2023/04/09/can-you-trust-a-compiler-to-optimize-your-code.html https://matklad.github.io/2023/04/09/can-you-trust-a-compile...

jesse__ 1y ago

I disagree pretty strongly with most of what you said, but I'd be very interested in seeing a Highway example and looking at the differences. Take a look through the comments, I left a link to the test bench I made, which contains all the code.

janwas 1y ago
Highway author here :) I'm curious what you disagree with, because it all sounds very sensible to me?
- jesse__ 1y ago
  There's a lot to discuss. First off, a number of statements are nonsense. Take, for example > you shouldn't be writing SIMD instructions directly unless you're writing a SIMD library or an optimizing compiler. Why would writing an optimizing compiler qualify as territory for directly writing SIMD code, but anything else is off the table? That makes no sense at all. Furthermore, I was writing a library. It's just embedded in my game engine. > Instead you should reach for one of the many available libraries This blanket statement is only true in a narrow set of circumstances. In my mind, it requires that you ship on multiple architectures and probably multiple compilers. If you have narrower constraints, it's extremely easy to write your own wrappers (like I did) and not take a dependency. A good trade IMO. Furthermore, someone's got to write the libraries, so doing it yourself as a learning exercise has value. > There are loads of libraries like this [...] and provide targeting for a vast trove of SIMD options without hand-writing for every option. The original commentor seems to be under the impression that using a SIMD library would somehow have produced a better result. The fact is, the library code is super fucking boring. I barely mentioned it in the article because it's basically just boilerplate an LLM could probably spit out, first try. The interesting part of the series is the observation that you can precompute a matrix of intermediates and look them up, instead of recomputing them in the hot loop, effectively trading memory bandwidth for less instructions. A good trade for this algorithm, which saturates the instruction pipelines. The thing the original commentor does get right is the notion that thinking about data layout is important. But, that has nothing to do with the library you're using .. you just have to do it. They seem to be conflating the use of a library with the act of writing wide code, as if you can't do one without the other, which is obviously false. > I was going to quickly rewrite the example in Highway .. Right. I'll believe this when I see it. I could pick it apart more, but.. I think you get my drift.