3 ms·

Though I have read a bit about simdjson I'm not super familiar with how it works, so please forgive me if this is a silly question. Since simdjson is where a l

Though I have read a bit about simdjson I'm not super familiar with how it works, so please forgive me if this is a silly question. Since simdjson is where a lot of the performance benefits come from here, would you need to compile the underlying C/C++ code on the machine that you're deploying on to make sure that simdjson is using the correct instruction set? Like what if the processor I'm compiling on has AVX-512 support, but the target machine for deployment doesn't? Does it just generate the machine code for all instruction sets and then can automatically choose to use the optimal instructios at runtime? Or does it just have to compile every time you pip install the package? I could see this being awkward if you're building a docker container perhaps, but maybe I'm just woefully uninformed.

knutzui 4y ago
The README mentions this: > Selects a CPU-tailored parser at runtime. No configuration needed. https://github.com/simdjson/simdjson https://github.com/simdjson/simdjson
- ZeroCool2u 4y ago
  Thank you, I missed that when I first looked at the README. Very cool if anyone else wants to take a look: https://github.com/simdjson/simdjson/blob/master/doc/implementation-selection.md https://github.com/simdjson/simdjson/blob/master/doc/impleme...
loeg 4y ago
It looks like the vast majority of the benefit (1200us -> 85us) comes from using WebSockets instead of REST (HTTP and maybe other protocol overhead). 85us beats GRPC in their tests, so it’s likely adequate for many applications.
rpep 4y ago
This isn't a silly question at all, and it's quite complicated. When compiling C/C++, with some compilers you are able to specify multiple architectures and provide a fat binary. It's been a few years now since I've worked on this sort of thing, but the Intel compiler for e.g. used to allow you to do something like this at the compilation stage -march=avx,avx2,avx512 And at runtime, your processor would use the most recent instruction set. In practice, you don't always want to do this, since you produce fat binaries - i.e. every function call that's got vector instructions will have multiple versions, and the file size goes up, library is slower to load, etc... I can't remember if there was an overhead too to each function call. So instead of doing this, what is also common is to compile N versions of the library (one per instruction set), and then load the appropriate version at runtime. This you can do with any compiler, and is indeed what Intel do themselves with the MKL library - if you look into the package for it, you can see that you end up with multiple shared libraries, each with a suffix saying which instruction set it supports (e.g. libmkl_avx.so, libmkl_avx2.so, libmkl_avx512.so). I'm not familiar with simdjson so not sure which approach it takes. Interestingly, if you use ARM processors, the SVE instruction set is designed so that you don't need to re-compile if a new processor family comes along with longer vector processing units.
- ZeroCool2u 4y ago
  Thanks, this is super helpful. I didn't know about that feature of the SVE instructions. I was curious and it looks like RISC-V takes it a step further even and is "Vector Length Agnostic"[1]. Pretty cool! [1]: https://gms.tf/riscv-vector.html https://gms.tf/riscv-vector.html

knutzui 4y ago

The README mentions this: > Selects a CPU-tailored parser at runtime. No configuration needed. https://github.com/simdjson/simdjson https://github.com/simdjson/simdjson

ZeroCool2u 4y ago
Thank you, I missed that when I first looked at the README. Very cool if anyone else wants to take a look: https://github.com/simdjson/simdjson/blob/master/doc/implementation-selection.md https://github.com/simdjson/simdjson/blob/master/doc/impleme...

loeg 4y ago

It looks like the vast majority of the benefit (1200us -> 85us) comes from using WebSockets instead of REST (HTTP and maybe other protocol overhead). 85us beats GRPC in their tests, so it’s likely adequate for many applications.

rpep 4y ago

This isn't a silly question at all, and it's quite complicated. When compiling C/C++, with some compilers you are able to specify multiple architectures and provide a fat binary. It's been a few years now since I've worked on this sort of thing, but the Intel compiler for e.g. used to allow you to do something like this at the compilation stage -march=avx,avx2,avx512 And at runtime, your processor would use the most recent instruction set. In practice, you don't always want to do this, since you produce fat binaries - i.e. every function call that's got vector instructions will have multiple versions, and the file size goes up, library is slower to load, etc... I can't remember if there was an overhead too to each function call. So instead of doing this, what is also common is to compile N versions of the library (one per instruction set), and then load the appropriate version at runtime. This you can do with any compiler, and is indeed what Intel do themselves with the MKL library - if you look into the package for it, you can see that you end up with multiple shared libraries, each with a suffix saying which instruction set it supports (e.g. libmkl_avx.so, libmkl_avx2.so, libmkl_avx512.so). I'm not familiar with simdjson so not sure which approach it takes. Interestingly, if you use ARM processors, the SVE instruction set is designed so that you don't need to re-compile if a new processor family comes along with longer vector processing units.

ZeroCool2u 4y ago
Thanks, this is super helpful. I didn't know about that feature of the SVE instructions. I was curious and it looks like RISC-V takes it a step further even and is "Vector Length Agnostic"[1]. Pretty cool! [1]: https://gms.tf/riscv-vector.html https://gms.tf/riscv-vector.html