3 ms·

Curious to think why they opted for Go and not C++ for the core engine, where there are lots of batched mathematical calculations. If you want to eventually le

Curious to think why they opted for Go and not C++ for the core engine, where there are lots of batched mathematical calculations. If you want to eventually leverage SIMD and GPGPU computation C++ is almost mandatory, languages like Rust/Nim/Zig are getting there but still behind in many areas.

anonymoushn 4y ago
> If you want to eventually leverage SIMD and GPGPU computation C++ is almost mandatory, languages like Rust/Nim/Zig are getting there but still behind in many areas. Is this mostly about GPGPU? I think you can just import and use SIMD intrinsics in Rust. Zig lets you do that and additionally features native vector types whose operations generate the appropriate SIMD instructions. There's nothing like Highway yet, but for many use cases it seems like a lot more trouble to use Highway than to write several implementations that use intrinsics directly.
- bedatadriven 4y ago
  The JVM is quite good at emitting SIMD instructions if you write straightforward loops with primitives. The forthcomming Vector API will open up more possibilities.
  - anonymoushn 4y ago
    I am less excited by auto vectorization than I once was after being repeatedly disappointed by LLVM's output, especially for string parsers. I hope it works well though.
- cyber_kinetist 4y ago
  Yeah mainly GPGPU. C++ still has an advantage in that there are quite some convenient SIMD libraries available (like Agner Fog’s vectorclass, and Google’s Highway that you have mentioned), but you can still write raw intrinsics in other languages (actually Zig has vector types baked in so you can use this instead). Though on second thought: for this particular company’s usecase, since they only need this for the server backend it would be best to stick to a certain hardware configuration and optimize the hell out of it. For example they can just stick to Xeon servers and use raw AVX512 intrinsics which can be particularly fast for number crunching. And since GPUs are incredibly expensive to host (and also has latency costs associated with it), it might be best to just stick with CPU SIMD.
Sirupsen 4y ago
Great question. The biggest reason is that Go was what the team was most familiar with when the engine moved from Javascript. The reason today is that the engine is 40K LOC of Go, so a full-blown rewrite would be a big deal. As the post goes into depth with, we have a lot of rope left before we need to get to SIMD/GPGPU. Once we hit those computational bottlenecks in Go, we're very open to changing our mind!
- melony 4y ago
  Go supports custom assembly, have you considered just polyfilling the SIMD functions manually?
  - Sirupsen 4y ago
    Yes, I didn't mean to imply we wouldn't go far to squeeze Go further! At some point the downside of the complexity of dropping down from Go to intrinsics, bypassing the GC, etc. might overwhelm the benefits of staying with Go. We've spent a while writing simulations like the ones in the post to understand Go's GC's behaviour under lots of pressure, and it won't be a bottleneck for a long time. I also think that doing all the pre-requisite work to make SIMD remotely possibly will yield _far_ more performance than SIMD itself.
    - phpnode 4y ago
      GC pressure is the cause of many of your competitors' problems. I'd be planning a move to an alternative sooner rather than later otherwise you'll run into the exact same problems that they have - after a certain point in the company's life it becomes practically impossible to fix
- trament 4y ago
  Now that you have decent data structures, why don't you use a library for arithmetic calculations on vectors? I would use MKL for this, at least if you're running on Intel machines, the vdMul function for your particular example. Even on AMD machines the MKL implementation will probably be faster than hand-written code.
deleted 4y ago
[deleted]

anonymoushn 4y ago

> If you want to eventually leverage SIMD and GPGPU computation C++ is almost mandatory, languages like Rust/Nim/Zig are getting there but still behind in many areas. Is this mostly about GPGPU? I think you can just import and use SIMD intrinsics in Rust. Zig lets you do that and additionally features native vector types whose operations generate the appropriate SIMD instructions. There's nothing like Highway yet, but for many use cases it seems like a lot more trouble to use Highway than to write several implementations that use intrinsics directly.

bedatadriven 4y ago
The JVM is quite good at emitting SIMD instructions if you write straightforward loops with primitives. The forthcomming Vector API will open up more possibilities.
- anonymoushn 4y ago
  I am less excited by auto vectorization than I once was after being repeatedly disappointed by LLVM's output, especially for string parsers. I hope it works well though.
cyber_kinetist 4y ago
Yeah mainly GPGPU. C++ still has an advantage in that there are quite some convenient SIMD libraries available (like Agner Fog’s vectorclass, and Google’s Highway that you have mentioned), but you can still write raw intrinsics in other languages (actually Zig has vector types baked in so you can use this instead). Though on second thought: for this particular company’s usecase, since they only need this for the server backend it would be best to stick to a certain hardware configuration and optimize the hell out of it. For example they can just stick to Xeon servers and use raw AVX512 intrinsics which can be particularly fast for number crunching. And since GPUs are incredibly expensive to host (and also has latency costs associated with it), it might be best to just stick with CPU SIMD.

Sirupsen 4y ago

Great question. The biggest reason is that Go was what the team was most familiar with when the engine moved from Javascript. The reason today is that the engine is 40K LOC of Go, so a full-blown rewrite would be a big deal. As the post goes into depth with, we have a lot of rope left before we need to get to SIMD/GPGPU. Once we hit those computational bottlenecks in Go, we're very open to changing our mind!

melony 4y ago
Go supports custom assembly, have you considered just polyfilling the SIMD functions manually?
- Sirupsen 4y ago
  Yes, I didn't mean to imply we wouldn't go far to squeeze Go further! At some point the downside of the complexity of dropping down from Go to intrinsics, bypassing the GC, etc. might overwhelm the benefits of staying with Go. We've spent a while writing simulations like the ones in the post to understand Go's GC's behaviour under lots of pressure, and it won't be a bottleneck for a long time. I also think that doing all the pre-requisite work to make SIMD remotely possibly will yield _far_ more performance than SIMD itself.
  - phpnode 4y ago
    GC pressure is the cause of many of your competitors' problems. I'd be planning a move to an alternative sooner rather than later otherwise you'll run into the exact same problems that they have - after a certain point in the company's life it becomes practically impossible to fix
trament 4y ago
Now that you have decent data structures, why don't you use a library for arithmetic calculations on vectors? I would use MKL for this, at least if you're running on Intel machines, the vdMul function for your particular example. Even on AMD machines the MKL implementation will probably be faster than hand-written code.

deleted 4y ago

[deleted]