6 ms·

Rustgo: Calling Rust from Go with near-zero overhead (2017)

185 points by telotortium 2y ago

gnabgib 2y ago
(2017) Discussions: 2017 (282 points, 68 comments) https://news.ycombinator.com/item?id=15017519 https://news.ycombinator.com/item?id=15017519 2019 (107 points, 37 comments) https://news.ycombinator.com/item?id=20600178 https://news.ycombinator.com/item?id=20600178
- deleted 2y ago
  [deleted]
zxilly 2y ago
There are many years since the article being written. I'm wondering if there is a better solution in 2024.
- ozgrakkurt 2y ago
  Doesn’t seem like there is. Probably because go developers want code to be pure go if possible
- mappu 2y ago
  Cgo overhead was made 5-30x faster in go1.21, and it was already within noise levels in this article, so i'd use that.
  - chabad360 2y ago
    Can you provide a reference for that? A quick search for me didn't bring up anything. I'm curious to read more.
    - lossolo 2y ago
      https://shane.ai/posts/cgo-performance-in-go1.21/ https://shane.ai/posts/cgo-performance-in-go1.21/
      - mananaysiempre 2y ago
        > Cgo calls take about 40ns, about the same time encoding/json takes to parse a single digit integer. As an aside, that sure feels like a lot of time to parse a single-digit integer. Not catastrophic by any means, but still, 100 to 200 cycles is a sign the program is definitely Doing Something. Perhaps that’s the memory allocator?
        ecnahc515 2y ago
        It's probably just because it's using reflection which is pretty slow in general. Lots of conditions/branches and boxing/unboxing of values.
- benmmurphy 2y ago
  they changed the ABI in 1.17 to pass arguments in registers instead of the stack: https://go.dev/doc/go1.17#compiler https://go.dev/doc/go1.17#compiler so if you used this solution you might not need to do the fixup anymore if the ABI matches.
deleted 2y ago
[deleted]
sitkack 2y ago
This is totally sick! I can't wait to go through these repro steps using new versions.
Havoc 2y ago
What is the benefit of this? I was under the impression that go isn't that far off from rust on most speed metrics being both compiled. So this adds complexity for what gain?
- Yoric 2y ago
  In benchmarks I've seen, Go is ~6 times slower. But generally speaking, the reason I prefer Rust to Go for most developments is its type system. I could see myself using Rust to handle business-specific/safety-critical logic and Go as a fast prototyping language to assemble them.
  - dewey 2y ago
    What part of the Go typing system is lacking for "business-specific/safety-critical logic"?
    - randomdata 2y ago
      It lacks expressiveness, which means you actually have to put some thought into your design to avoid butting heads with it. You can't throw up garbage and then keep massaging the types until you can make whatever gobbledygook you came up with fit. Business/safety critical logic tends to get complicated, and, at least in the case of business software, is more than likely is venturing into unexplored territory, so when you're not very skilled at programming finding the right design can become an insurmountable challenge. A more expressive type system offers a bandaid to work around that.
      - mr90210 2y ago
        [flagged]
        randomdata 2y ago
        Strictly speaking, that is true. Even the worst code ever written had some thought put into it. But the "actually put some thought" was meant to imply more than putting down the first characters that spew out; to consider the design more holistically and carefully. But you are right to not take anything that software says seriously. Seriousness is for interactions with humans.
nickcw 2y ago
That was a great read. All that linker wrangling is sure to break on the next version of go/rust/linker isn't it? I wonder if it would have been easier to disassemble the rust binary and turn it into Go assembly and use that. That would need a fairly complicated program to process the binary back into assembler. Maybe getting the rust compiler to output assembly and processing into Go assembly would be the way. Using Go assembly would save fighting with the linker, be more likely to survive upgrades and it would be cross platform (well at least on platforms with the same CPU arch).
- neonsunset 2y ago
  Imagine picking a language with terrible FFI overhead and weak compiler only to fight these two worsts aspects of it in an attempt to fix them. C# with zero-cost FFI, none of the performance penalty and ability to statically link everything together is a strictly better choice. Saner type system and syntax too.
  - kgeist 2y ago
    C#'s FFI is kind of zero-cost with blittable types (int, float). You still need to do marshalling for anything more complex (strings was a common issue on Linux, UTF8<=>UTF16), also memory pinning. Last time I checked, it also notifies the GC the thread entered native code (can't be preempted, the stack needs to be scanned conservatively etc.) After exiting a native function IIRC there's a safepoint check. I remember years ago it was common knowledge that P/Invoke is not suitable for calling hot functions in a loop, you had to create native helper functions which make all the calls in one go. Maybe it has changed?
    - neonsunset 2y ago
      Memory pinning is practically free and GC does quite a bit of work to further minimize its impact on throughput, common practice is to use stack buffers or natively allocated memory for marhsalled or other data anyway (remember - free FFI, so you can always do malloc and free which marshallers do). In practice UTF-16<->UTF-8 conversion rarely shows up on flamegraph as it turns out built-in transcoding is very fast and does not leave dangling data that GC needs to clean up. It is also not as frequently needed - you can just get a byte* for free out of "hello, world"u8 and pass it without any extra operations (the binding generator will do that for you). On top of that, complex blittable structures are first and foremost expressible in C# - you can easily have C binary tree, or an array of arrays with byte**. Performance-sensitive code that does interop heavily exploits that to give actual C-like experience. Also short-lived FFI calls do not need to notify GC (which is also cheap it toggles a boolean), libraries that care about extra last nanosecond annotate them with `[SuppressGCTransition]` which further streamlines assembly around FFI call. For performing FFI in a loop - the cost comes from the same reason a non-inlineable cross-compilation-unit calls are expensive within C++. Because we're talking plain call level of overhead, which is by definition as cheap as FFI gets. Of course that can still be a bottleneck if you have a small function beyond FFI boundary within a hot loop. In that case you might as well port it to C# to make it inlineable which is going to be faster, or have an FFI call for a batched operation. Try `dotnet publish -o . -p:StripSymbols=false`ing this: https://github.com/U8String/U8String/tree/main/Examples/Interop/CSharp https://github.com/U8String/U8String/tree/main/Examples/Inte... and then disassembling it with Ghidra. You will be positively impressed with how codegen looks - there will be simple direct calls into Rust with single bool checks for a potential GC poll after them (and compiler will merge those too after multiple consecutive calls).
odanalysis 2y ago
Does this mean you could now use gokrazy to run rust apps on an sbc with better startup speeds than linux??
- sureglymop 2y ago
  If you're calling rust from go, wouldn't you still have the go startup time before you could call the rust code?
yutijke 2y ago
https://github.com/petermattis/fastcgo https://github.com/petermattis/fastcgo, which is now 7 years old seems to do something similar without the need to about obscure CGO FFI configuration. It also seems to be more generally applicable for any language with C interop. There had been an issue for having something similar in the language itself - https://github.com/golang/go/issues/42469 https://github.com/golang/go/issues/42469, but the Golang compiler team rejected it. If you have followed similar discussions around this with the Golang compiler team, you will notice a pattern of interaction that strongly indicates that they are very much opposed to ever accepting this into the compiler.
- dontlaugh 2y ago
  fastcgo links to rustgo
- deleted 2y ago
  [deleted]
roundup 2y ago
[dead]
ethegwo 2y ago
it makes me think about https://news.ycombinator.com/item?id=41117749 https://news.ycombinator.com/item?id=41117749 it is the mirror of this project: effient wal to call Go from Rust
hardwaregeek 2y ago
lol I remember looking at this when doing our major port from Go to Rust. I noped out once I saw the raw assembly portion. Great for a side project but probably not the move for a cross platform binary run by thousands of people. Still a very cool post and the literature for Go/Rust interop is def lacking

185 points by telotortium 2y ago

(2017) Discussions: 2017 (282 points, 68 comments) https://news.ycombinator.com/item?id=15017519 https://news.ycombinator.com/item?id=15017519 2019 (107 points, 37 comments) https://news.ycombinator.com/item?id=20600178 https://news.ycombinator.com/item?id=20600178

deleted 2y ago
[deleted]

zxilly 2y ago

There are many years since the article being written. I'm wondering if there is a better solution in 2024.

ozgrakkurt 2y ago
Doesn’t seem like there is. Probably because go developers want code to be pure go if possible
mappu 2y ago
Cgo overhead was made 5-30x faster in go1.21, and it was already within noise levels in this article, so i'd use that.
- chabad360 2y ago
  Can you provide a reference for that? A quick search for me didn't bring up anything. I'm curious to read more.
  - lossolo 2y ago
    https://shane.ai/posts/cgo-performance-in-go1.21/ https://shane.ai/posts/cgo-performance-in-go1.21/
    - mananaysiempre 2y ago
      > Cgo calls take about 40ns, about the same time encoding/json takes to parse a single digit integer. As an aside, that sure feels like a lot of time to parse a single-digit integer. Not catastrophic by any means, but still, 100 to 200 cycles is a sign the program is definitely Doing Something. Perhaps that’s the memory allocator?
      - ecnahc515 2y ago
        It's probably just because it's using reflection which is pretty slow in general. Lots of conditions/branches and boxing/unboxing of values.
benmmurphy 2y ago
they changed the ABI in 1.17 to pass arguments in registers instead of the stack: https://go.dev/doc/go1.17#compiler https://go.dev/doc/go1.17#compiler so if you used this solution you might not need to do the fixup anymore if the ABI matches.

deleted 2y ago

[deleted]

sitkack 2y ago

This is totally sick! I can't wait to go through these repro steps using new versions.

Havoc 2y ago

What is the benefit of this? I was under the impression that go isn't that far off from rust on most speed metrics being both compiled. So this adds complexity for what gain?

Yoric 2y ago
In benchmarks I've seen, Go is ~6 times slower. But generally speaking, the reason I prefer Rust to Go for most developments is its type system. I could see myself using Rust to handle business-specific/safety-critical logic and Go as a fast prototyping language to assemble them.
- dewey 2y ago
  What part of the Go typing system is lacking for "business-specific/safety-critical logic"?
  - randomdata 2y ago
    It lacks expressiveness, which means you actually have to put some thought into your design to avoid butting heads with it. You can't throw up garbage and then keep massaging the types until you can make whatever gobbledygook you came up with fit. Business/safety critical logic tends to get complicated, and, at least in the case of business software, is more than likely is venturing into unexplored territory, so when you're not very skilled at programming finding the right design can become an insurmountable challenge. A more expressive type system offers a bandaid to work around that.
    - mr90210 2y ago
      [flagged]
      - randomdata 2y ago
        Strictly speaking, that is true. Even the worst code ever written had some thought put into it. But the "actually put some thought" was meant to imply more than putting down the first characters that spew out; to consider the design more holistically and carefully. But you are right to not take anything that software says seriously. Seriousness is for interactions with humans.

nickcw 2y ago

That was a great read. All that linker wrangling is sure to break on the next version of go/rust/linker isn't it? I wonder if it would have been easier to disassemble the rust binary and turn it into Go assembly and use that. That would need a fairly complicated program to process the binary back into assembler. Maybe getting the rust compiler to output assembly and processing into Go assembly would be the way. Using Go assembly would save fighting with the linker, be more likely to survive upgrades and it would be cross platform (well at least on platforms with the same CPU arch).

neonsunset 2y ago
Imagine picking a language with terrible FFI overhead and weak compiler only to fight these two worsts aspects of it in an attempt to fix them. C# with zero-cost FFI, none of the performance penalty and ability to statically link everything together is a strictly better choice. Saner type system and syntax too.
- kgeist 2y ago
  C#'s FFI is kind of zero-cost with blittable types (int, float). You still need to do marshalling for anything more complex (strings was a common issue on Linux, UTF8<=>UTF16), also memory pinning. Last time I checked, it also notifies the GC the thread entered native code (can't be preempted, the stack needs to be scanned conservatively etc.) After exiting a native function IIRC there's a safepoint check. I remember years ago it was common knowledge that P/Invoke is not suitable for calling hot functions in a loop, you had to create native helper functions which make all the calls in one go. Maybe it has changed?
  - neonsunset 2y ago
    Memory pinning is practically free and GC does quite a bit of work to further minimize its impact on throughput, common practice is to use stack buffers or natively allocated memory for marhsalled or other data anyway (remember - free FFI, so you can always do malloc and free which marshallers do). In practice UTF-16<->UTF-8 conversion rarely shows up on flamegraph as it turns out built-in transcoding is very fast and does not leave dangling data that GC needs to clean up. It is also not as frequently needed - you can just get a byte* for free out of "hello, world"u8 and pass it without any extra operations (the binding generator will do that for you). On top of that, complex blittable structures are first and foremost expressible in C# - you can easily have C binary tree, or an array of arrays with byte**. Performance-sensitive code that does interop heavily exploits that to give actual C-like experience. Also short-lived FFI calls do not need to notify GC (which is also cheap it toggles a boolean), libraries that care about extra last nanosecond annotate them with `[SuppressGCTransition]` which further streamlines assembly around FFI call. For performing FFI in a loop - the cost comes from the same reason a non-inlineable cross-compilation-unit calls are expensive within C++. Because we're talking plain call level of overhead, which is by definition as cheap as FFI gets. Of course that can still be a bottleneck if you have a small function beyond FFI boundary within a hot loop. In that case you might as well port it to C# to make it inlineable which is going to be faster, or have an FFI call for a batched operation. Try `dotnet publish -o . -p:StripSymbols=false`ing this: https://github.com/U8String/U8String/tree/main/Examples/Interop/CSharp https://github.com/U8String/U8String/tree/main/Examples/Inte... and then disassembling it with Ghidra. You will be positively impressed with how codegen looks - there will be simple direct calls into Rust with single bool checks for a potential GC poll after them (and compiler will merge those too after multiple consecutive calls).

odanalysis 2y ago

Does this mean you could now use gokrazy to run rust apps on an sbc with better startup speeds than linux??

sureglymop 2y ago
If you're calling rust from go, wouldn't you still have the go startup time before you could call the rust code?

yutijke 2y ago

https://github.com/petermattis/fastcgo https://github.com/petermattis/fastcgo, which is now 7 years old seems to do something similar without the need to about obscure CGO FFI configuration. It also seems to be more generally applicable for any language with C interop. There had been an issue for having something similar in the language itself - https://github.com/golang/go/issues/42469 https://github.com/golang/go/issues/42469, but the Golang compiler team rejected it. If you have followed similar discussions around this with the Golang compiler team, you will notice a pattern of interaction that strongly indicates that they are very much opposed to ever accepting this into the compiler.

dontlaugh 2y ago
fastcgo links to rustgo
deleted 2y ago
[deleted]

roundup 2y ago

[dead]

ethegwo 2y ago

it makes me think about https://news.ycombinator.com/item?id=41117749 https://news.ycombinator.com/item?id=41117749 it is the mirror of this project: effient wal to call Go from Rust

hardwaregeek 2y ago

lol I remember looking at this when doing our major port from Go to Rust. I noped out once I saw the raw assembly portion. Great for a side project but probably not the move for a cross platform binary run by thousands of people. Still a very cool post and the literature for Go/Rust interop is def lacking