5 ms·

Tangentially related: Is it currently possible to have interop between Go and Rust? I remember seeing someone achieving it with Zig in the middle but can’t for

Tangentially related: Is it currently possible to have interop between Go and Rust? I remember seeing someone achieving it with Zig in the middle but can’t for the sake of me find it. Have some legacy Rust code (what??) that I’m hoping to slowly port to Go piece by piece

apendleton 3y ago
If you want to call from Go into Rust, you can declare any Rust function as `extern "C"` and then call it the same way you would call C from Go. Not sure about going the other way.
duped 3y ago
It's usually unwise to mix managed and unmanaged memory since the managed code needs to be able to own the memory its freeing and moving whereas the unmanaged code needs to reason about when memory is freed or moved. cgo (and other variants) let you mix FFI calls into unmanaged memory from managed code in Go, but you pay a penalty for it. In language implementations where GC isn't shared by the different languages calling each other you're always going to have this problem. Mixing managed/unmanaged code is both an old idea and actively researched. It's almost always a terrible idea to call into managed code from unmanaged code unless you're working directly with an embedded runtime that's been designed for it. And when you do, there's usually a serialization layer in between.
- neonsunset 3y ago
  Mixing managed and unmanaged code being an issue is simply not true in programming in general. It may be an issue in Go or Java, but it just isn't in C# or Swift. Calling `write` in C# on Unix is as easy as the following snippet and has almost no overhead: var text = "Hello, World!\n"u8; Interop.Write(1, text, text.Length); static unsafe partial class Interop { [LibraryImport("libc", EntryPoint = "write")] public static partial void Write( nint fd, ReadOnlySpan<byte> buffer, nint length); } In addition, unmanaged->managed calls are also rarely an issue, both via function pointers and plain C exports if you build a binary with NativeAOT: public static class Exports { [UnmanagedCallersOnly(EntryPoint = "sum")] public static nint Sum(nint a, nint b) => a + b; } It is indeed true that more complex scenarios may require some form of bespoke embedding/hosting of the runtime, but that is more of a peculiarity of Go and Java, not an actual technical limitation.
  - jcranmer 3y ago
    That's not the direction being talked about here. Try calling the C# method from C or C++ or Rust. (I somewhat recently did try setting up mono to be able to do this... it wasn't fun.)
    - neonsunset 3y ago
      What you may have been looking for is these: - https://learn.microsoft.com/en-us/dotnet/core/deploying/native-aot/interop#native-exports https://learn.microsoft.com/en-us/dotnet/core/deploying/nati... - https://github.com/dotnet/samples/blob/main/core/nativeaot/NativeLibrary/README.md https://github.com/dotnet/samples/blob/main/core/nativeaot/N... With that said, Mono has been a staple choice for embedding in game-script style scenarios, in particular, because of the ability to directly call its methods inside (provided the caller honors the calling convention correctly), but it has been slowly becoming more of a liability as you are missing out on a lot of performance by not hosting CoreCLR instead. For .dll/.so/.dylib's, it is easier and often better to just build a native library with naot instead (the links above, you can also produce statically linkable binaries but it might have issues on e.g. macOS which has...not the most reliable linker that likes to take breaking changes). This type of library works in almost every scenario a library implemented in C/C++/Rust with C exports does. For example, here someone implemented a hello-world demonstration of using C# to write an OBS plugin: https://sharovarskyi.com/blog/posts/dotnet-obs-plugin-with-nativeaot/ https://sharovarskyi.com/blog/posts/dotnet-obs-plugin-with-n... Using the exports boils down to just this https://github.com/kostya9/DotnetObsPluginWithNativeAOT/blob/main/ObsPlugin.cs https://github.com/kostya9/DotnetObsPluginWithNativeAOT/blob... and specifying correct build flags.
      - duped 3y ago
        I haven't been looking for those because I don't work with .NET. Regardless, what you're linking still needs callers and callees to agree on calling convention and special binding annotations across FFI boundaries which isn't particularly interesting from the perspective of language implementation like the promises of Graal or WASM + GC + component model.
- zozbot234 3y ago
  > It's usually unwise to mix managed and unmanaged memory Broadly stated, you can achieve this by marking a managed object as a GC root whenever it's to be referenced by unmanaged code (so that it won't be freed or moved in that case) and adding finalizers whenever managed objects own or hold refcounted references over unmanaged memory (so that the unmanaged code can reason about these objects being freed). But yes, it's a bit fiddly.
100k 3y ago
Yes, you can use CGO to call Rust functions using extern "C" FFI. I gave a talk about how we use it for GitHub code search at RustConf 2023 (https://www.youtube.com/watch?v=KYdlqhb267c https://www.youtube.com/watch?v=KYdlqhb267c) and afterwards I talked to some other folks (like 1Password) who are doing similar things. It's not a lot of fun because moving types across the C interop boundary is tedious, but it is possible and allows code reuse.
neonsunset 3y ago
You have to go through C bindings, but FFI is very far from being Go's strongest suit (if we don't count Cgo), so if that's what interests you, it might be better to explore a different language.
mrits 3y ago
I have to use Rust and Swift quite a bit. I basically just landed on sending a byte array of serialized protobufs back and forth with cookie cutter function calls. If this is your full time job I can see how you might think that is lame, but I really got tired of coming back to the code every few weeks and not remembering how to do anything.
Voultapher 3y ago
If you want a particularity cursed example, I've recently called Go code from Rust via C in the middle, including passing a Rust closure with state into the Go code as callback into a Go stdlib function, including panic unwinding from inside the Rust closure https://github.com/Voultapher/sort-research-rs/commit/df6c91c664f262f69061cb45d4520d0df295610c https://github.com/Voultapher/sort-research-rs/commit/df6c91....

apendleton 3y ago

If you want to call from Go into Rust, you can declare any Rust function as `extern "C"` and then call it the same way you would call C from Go. Not sure about going the other way.

duped 3y ago

It's usually unwise to mix managed and unmanaged memory since the managed code needs to be able to own the memory its freeing and moving whereas the unmanaged code needs to reason about when memory is freed or moved. cgo (and other variants) let you mix FFI calls into unmanaged memory from managed code in Go, but you pay a penalty for it. In language implementations where GC isn't shared by the different languages calling each other you're always going to have this problem. Mixing managed/unmanaged code is both an old idea and actively researched. It's almost always a terrible idea to call into managed code from unmanaged code unless you're working directly with an embedded runtime that's been designed for it. And when you do, there's usually a serialization layer in between.

neonsunset 3y ago
Mixing managed and unmanaged code being an issue is simply not true in programming in general. It may be an issue in Go or Java, but it just isn't in C# or Swift. Calling `write` in C# on Unix is as easy as the following snippet and has almost no overhead: var text = "Hello, World!\n"u8; Interop.Write(1, text, text.Length); static unsafe partial class Interop { [LibraryImport("libc", EntryPoint = "write")] public static partial void Write( nint fd, ReadOnlySpan<byte> buffer, nint length); } In addition, unmanaged->managed calls are also rarely an issue, both via function pointers and plain C exports if you build a binary with NativeAOT: public static class Exports { [UnmanagedCallersOnly(EntryPoint = "sum")] public static nint Sum(nint a, nint b) => a + b; } It is indeed true that more complex scenarios may require some form of bespoke embedding/hosting of the runtime, but that is more of a peculiarity of Go and Java, not an actual technical limitation.
- jcranmer 3y ago
  That's not the direction being talked about here. Try calling the C# method from C or C++ or Rust. (I somewhat recently did try setting up mono to be able to do this... it wasn't fun.)
  - neonsunset 3y ago
    What you may have been looking for is these: - https://learn.microsoft.com/en-us/dotnet/core/deploying/native-aot/interop#native-exports https://learn.microsoft.com/en-us/dotnet/core/deploying/nati... - https://github.com/dotnet/samples/blob/main/core/nativeaot/NativeLibrary/README.md https://github.com/dotnet/samples/blob/main/core/nativeaot/N... With that said, Mono has been a staple choice for embedding in game-script style scenarios, in particular, because of the ability to directly call its methods inside (provided the caller honors the calling convention correctly), but it has been slowly becoming more of a liability as you are missing out on a lot of performance by not hosting CoreCLR instead. For .dll/.so/.dylib's, it is easier and often better to just build a native library with naot instead (the links above, you can also produce statically linkable binaries but it might have issues on e.g. macOS which has...not the most reliable linker that likes to take breaking changes). This type of library works in almost every scenario a library implemented in C/C++/Rust with C exports does. For example, here someone implemented a hello-world demonstration of using C# to write an OBS plugin: https://sharovarskyi.com/blog/posts/dotnet-obs-plugin-with-nativeaot/ https://sharovarskyi.com/blog/posts/dotnet-obs-plugin-with-n... Using the exports boils down to just this https://github.com/kostya9/DotnetObsPluginWithNativeAOT/blob/main/ObsPlugin.cs https://github.com/kostya9/DotnetObsPluginWithNativeAOT/blob... and specifying correct build flags.
    - duped 3y ago
      I haven't been looking for those because I don't work with .NET. Regardless, what you're linking still needs callers and callees to agree on calling convention and special binding annotations across FFI boundaries which isn't particularly interesting from the perspective of language implementation like the promises of Graal or WASM + GC + component model.
zozbot234 3y ago
> It's usually unwise to mix managed and unmanaged memory Broadly stated, you can achieve this by marking a managed object as a GC root whenever it's to be referenced by unmanaged code (so that it won't be freed or moved in that case) and adding finalizers whenever managed objects own or hold refcounted references over unmanaged memory (so that the unmanaged code can reason about these objects being freed). But yes, it's a bit fiddly.

100k 3y ago

Yes, you can use CGO to call Rust functions using extern "C" FFI. I gave a talk about how we use it for GitHub code search at RustConf 2023 (https://www.youtube.com/watch?v=KYdlqhb267c https://www.youtube.com/watch?v=KYdlqhb267c) and afterwards I talked to some other folks (like 1Password) who are doing similar things. It's not a lot of fun because moving types across the C interop boundary is tedious, but it is possible and allows code reuse.

neonsunset 3y ago

You have to go through C bindings, but FFI is very far from being Go's strongest suit (if we don't count Cgo), so if that's what interests you, it might be better to explore a different language.

mrits 3y ago

I have to use Rust and Swift quite a bit. I basically just landed on sending a byte array of serialized protobufs back and forth with cookie cutter function calls. If this is your full time job I can see how you might think that is lame, but I really got tired of coming back to the code every few weeks and not remembering how to do anything.

Voultapher 3y ago

If you want a particularity cursed example, I've recently called Go code from Rust via C in the middle, including passing a Rust closure with state into the Go code as callback into a Go stdlib function, including panic unwinding from inside the Rust closure https://github.com/Voultapher/sort-research-rs/commit/df6c91c664f262f69061cb45d4520d0df295610c https://github.com/Voultapher/sort-research-rs/commit/df6c91....