3 ms·

My experience micro optimizing things is that even without SIMD, most software can get at least a 5x in performance. With SIMD, you can often get 50x improvemen

My experience micro optimizing things is that even without SIMD, most software can get at least a 5x in performance. With SIMD, you can often get 50x improvements. The reason why people thing "Even if you do, you are only going to get a few percentage points." is because it generally takes 5-50x the developer time to optimize such code. If it takes half a day to write naive code to do something like validate utf8, it probably takes ~25 workdays to make a fast SIMD version. If you instead spend an extra half a day, there is a good chance you get a 10-50% speedup using normal code.

mumumu 4y ago
This is true on a few streaming application (such as parsing). And most of the speedup is because of tricks to avoid doing beaches. There is a great blog post from one of the authors of JSON SIMD discussing this. I'm on mobile, there is a link for the blog post on the simd JSON github repository.
- mumumu 4y ago
  *avoid branches The blog post I mentioned: Paper: Parsing Gigabytes of JSON per Second https://branchfree.org/2019/02/25/paper-parsing-gigabytes-of-json-per-second/ https://branchfree.org/2019/02/25/paper-parsing-gigabytes-of... Another related post from Lemire: Ridiculously fast unicode (UTF-8) validation https://lemire.me/blog/2020/10/20/ridiculously-fast-unicode-utf-8-validation/ https://lemire.me/blog/2020/10/20/ridiculously-fast-unicode-... Those algorithms are fast. But to put them in perspective. A single x86 CPU can write 64B per cycle. At 5GHz, the theorical maximum bandwidth is 320 GBps. IIRC, the read bandwidth is twice that. There are others botlenecks, and is very hard to write code that writes at every cycle. A interesting consequence, is that the theorical maximum bandwidth is logarithmical to the number of cycles. Again, talking about branchless streaming application.
bee_rider 4y ago
It also can we weird and non-obvious. For example depending on the instruction mix and hardware it might not be worth getting dinged by AVX-512 clocks. And if you are, say, writing the UTF validation code as a library (more favorable to put effort into a library!) you might not know where the code is being used, so you might not even know the instruction mix…

mumumu 4y ago

This is true on a few streaming application (such as parsing). And most of the speedup is because of tricks to avoid doing beaches. There is a great blog post from one of the authors of JSON SIMD discussing this. I'm on mobile, there is a link for the blog post on the simd JSON github repository.

mumumu 4y ago
*avoid branches The blog post I mentioned: Paper: Parsing Gigabytes of JSON per Second https://branchfree.org/2019/02/25/paper-parsing-gigabytes-of-json-per-second/ https://branchfree.org/2019/02/25/paper-parsing-gigabytes-of... Another related post from Lemire: Ridiculously fast unicode (UTF-8) validation https://lemire.me/blog/2020/10/20/ridiculously-fast-unicode-utf-8-validation/ https://lemire.me/blog/2020/10/20/ridiculously-fast-unicode-... Those algorithms are fast. But to put them in perspective. A single x86 CPU can write 64B per cycle. At 5GHz, the theorical maximum bandwidth is 320 GBps. IIRC, the read bandwidth is twice that. There are others botlenecks, and is very hard to write code that writes at every cycle. A interesting consequence, is that the theorical maximum bandwidth is logarithmical to the number of cycles. Again, talking about branchless streaming application.

bee_rider 4y ago

It also can we weird and non-obvious. For example depending on the instruction mix and hardware it might not be worth getting dinged by AVX-512 clocks. And if you are, say, writing the UTF validation code as a library (more favorable to put effort into a library!) you might not know where the code is being used, so you might not even know the instruction mix…