5 ms·

It's usually unwise to mix managed and unmanaged memory since the managed code needs to be able to own the memory its freeing and moving whereas the unmanaged c

by duped 3y ago

It's usually unwise to mix managed and unmanaged memory since the managed code needs to be able to own the memory its freeing and moving whereas the unmanaged code needs to reason about when memory is freed or moved. cgo (and other variants) let you mix FFI calls into unmanaged memory from managed code in Go, but you pay a penalty for it. In language implementations where GC isn't shared by the different languages calling each other you're always going to have this problem. Mixing managed/unmanaged code is both an old idea and actively researched. It's almost always a terrible idea to call into managed code from unmanaged code unless you're working directly with an embedded runtime that's been designed for it. And when you do, there's usually a serialization layer in between.

neonsunset 3y ago
Mixing managed and unmanaged code being an issue is simply not true in programming in general. It may be an issue in Go or Java, but it just isn't in C# or Swift. Calling `write` in C# on Unix is as easy as the following snippet and has almost no overhead: var text = "Hello, World!\n"u8; Interop.Write(1, text, text.Length); static unsafe partial class Interop { [LibraryImport("libc", EntryPoint = "write")] public static partial void Write( nint fd, ReadOnlySpan<byte> buffer, nint length); } In addition, unmanaged->managed calls are also rarely an issue, both via function pointers and plain C exports if you build a binary with NativeAOT: public static class Exports { [UnmanagedCallersOnly(EntryPoint = "sum")] public static nint Sum(nint a, nint b) => a + b; } It is indeed true that more complex scenarios may require some form of bespoke embedding/hosting of the runtime, but that is more of a peculiarity of Go and Java, not an actual technical limitation.
- jcranmer 3y ago
  That's not the direction being talked about here. Try calling the C# method from C or C++ or Rust. (I somewhat recently did try setting up mono to be able to do this... it wasn't fun.)
  - neonsunset 3y ago
    What you may have been looking for is these: - https://learn.microsoft.com/en-us/dotnet/core/deploying/native-aot/interop#native-exports https://learn.microsoft.com/en-us/dotnet/core/deploying/nati... - https://github.com/dotnet/samples/blob/main/core/nativeaot/NativeLibrary/README.md https://github.com/dotnet/samples/blob/main/core/nativeaot/N... With that said, Mono has been a staple choice for embedding in game-script style scenarios, in particular, because of the ability to directly call its methods inside (provided the caller honors the calling convention correctly), but it has been slowly becoming more of a liability as you are missing out on a lot of performance by not hosting CoreCLR instead. For .dll/.so/.dylib's, it is easier and often better to just build a native library with naot instead (the links above, you can also produce statically linkable binaries but it might have issues on e.g. macOS which has...not the most reliable linker that likes to take breaking changes). This type of library works in almost every scenario a library implemented in C/C++/Rust with C exports does. For example, here someone implemented a hello-world demonstration of using C# to write an OBS plugin: https://sharovarskyi.com/blog/posts/dotnet-obs-plugin-with-nativeaot/ https://sharovarskyi.com/blog/posts/dotnet-obs-plugin-with-n... Using the exports boils down to just this https://github.com/kostya9/DotnetObsPluginWithNativeAOT/blob/main/ObsPlugin.cs https://github.com/kostya9/DotnetObsPluginWithNativeAOT/blob... and specifying correct build flags.
    - duped 3y ago
      I haven't been looking for those because I don't work with .NET. Regardless, what you're linking still needs callers and callees to agree on calling convention and special binding annotations across FFI boundaries which isn't particularly interesting from the perspective of language implementation like the promises of Graal or WASM + GC + component model.
      - neonsunset 3y ago
        There is no free lunch. WASM just means another lowest common denominator abstraction for FFI. I'm also looking forward to WASM getting actually good so .NET could properly target it (because shipping WASM-compiled GC is really, really painful, it works acceptably today, but could be better). Current WasmGC spec is pretty much unusable by any language that has non-primitive GC implementation. Just please don't run WASM on the server, we're already getting diminishing generational performance gains in hardware, no need to reduce them further. The exports in the examples follow C ABI with respective OS/ISA-specific calling convention.
- duped 3y ago
  There are more managed langauges than Go, Java, and C#. Swift (and Objective C with ARC) are a bit different in that they don't use mark and sweep/generational GCs for automatic memory management so it's significantly less of an issue. Compare with Lua, Python, JS, etc where there's a serialization boundary between the two. But I stand by what I said. It's generally unwise to mix the two, particularly calling unmanaged code from managed code. I wouldn't say it's "not a problem" because there are very few environments where you don't pay some cost for mixing and matching between managed/unmanaged code, and the environments designed around it are built from first principles to support it, like .NET. More interesting to me are Graal and WASM (once GC support lands) which should make it much easier to deal with.
- meindnoch 3y ago
  Swift is not a "managed" (i.e. GC) language.
  - pjmlp 3y ago
    Reference counting is a GC algorithm in any decent CS book. "A Unified Theory of Garbage Collection" https://courses.cs.washington.edu/courses/cse590p/05au/p50-bacon.pdf https://courses.cs.washington.edu/courses/cse590p/05au/p50-b...
    - meindnoch 3y ago
      I was expecting this pedantic comment... If refcounting makes a language "managed", then C++ with shared_ptr is also "managed". _______ The charitable interpretation is that OP was likely referring to the issues when calling into a language with a relocating GC (because you need to tell the GC not to move objects while you're working with them), which Swift is not.
      - neonsunset 3y ago
        Swift has just as much concerns for its structs and classes passing across FFI in terms of marshalling/unmarshalling and ensuring the ARC-unaware code performs either manual retain/release calls or adapts them to whatever other mechanism of memory management of the callee. One of the comments here mentions that Swift has its own stable ABI, which exposes richer type system, so it does stand out in terms of interop (.NET 9 will add support for it natively (library evolution ABI) without having to go through C calls or C "glue" code on iOS and macOS, maybe the first non-Swift/ObjC platform to do so?). Object pinning in .NET is only a part of equation and at this point far from the biggest concern (it just works, like it did 15 years ago, maybe it's a matter of fuss elsewhere?).
      - pjmlp 3y ago
        Nope, because that is a library class without any language support. The pedantic comment is a synonymous with proper education instead of street urban myths.
- pjmlp 3y ago
  Except that is only true since those attributes were introduced in recent .NET versions, and it doesn't account for COM marshaling issues. Plenty of .NET code still using the old ways that isn't going to be rewritten, either for these attributes, or the new Cs/WinRT, or the new Core COM interop, which doesn't support all COM use cases anyway.
  - neonsunset 3y ago
    Code written for .NET Framework is completely irrelevant to conversation since it does not evaluate it. You should treat it as dead and move on because it does not impact what .NET can or can’t do. There is no point to bring up “No, but 10 years ago it was different”. So what? It’s not 2014 anymore.
    - pjmlp 3y ago
      My remarks also apply to modern .NET, as those improvements were introduced in .NET 6 and .NET 8, and require a code rewrite to adopt them, instead of the old ways which are also available, in your blind advocacy you happened to miss out. Very few code gets written from scratch unless we are talking about startups.
zozbot234 3y ago
> It's usually unwise to mix managed and unmanaged memory Broadly stated, you can achieve this by marking a managed object as a GC root whenever it's to be referenced by unmanaged code (so that it won't be freed or moved in that case) and adding finalizers whenever managed objects own or hold refcounted references over unmanaged memory (so that the unmanaged code can reason about these objects being freed). But yes, it's a bit fiddly.

by duped 3y ago

neonsunset 3y ago

Mixing managed and unmanaged code being an issue is simply not true in programming in general. It may be an issue in Go or Java, but it just isn't in C# or Swift. Calling `write` in C# on Unix is as easy as the following snippet and has almost no overhead: var text = "Hello, World!\n"u8; Interop.Write(1, text, text.Length); static unsafe partial class Interop { [LibraryImport("libc", EntryPoint = "write")] public static partial void Write( nint fd, ReadOnlySpan<byte> buffer, nint length); } In addition, unmanaged->managed calls are also rarely an issue, both via function pointers and plain C exports if you build a binary with NativeAOT: public static class Exports { [UnmanagedCallersOnly(EntryPoint = "sum")] public static nint Sum(nint a, nint b) => a + b; } It is indeed true that more complex scenarios may require some form of bespoke embedding/hosting of the runtime, but that is more of a peculiarity of Go and Java, not an actual technical limitation.

jcranmer 3y ago
That's not the direction being talked about here. Try calling the C# method from C or C++ or Rust. (I somewhat recently did try setting up mono to be able to do this... it wasn't fun.)
- neonsunset 3y ago
  What you may have been looking for is these: - https://learn.microsoft.com/en-us/dotnet/core/deploying/native-aot/interop#native-exports https://learn.microsoft.com/en-us/dotnet/core/deploying/nati... - https://github.com/dotnet/samples/blob/main/core/nativeaot/NativeLibrary/README.md https://github.com/dotnet/samples/blob/main/core/nativeaot/N... With that said, Mono has been a staple choice for embedding in game-script style scenarios, in particular, because of the ability to directly call its methods inside (provided the caller honors the calling convention correctly), but it has been slowly becoming more of a liability as you are missing out on a lot of performance by not hosting CoreCLR instead. For .dll/.so/.dylib's, it is easier and often better to just build a native library with naot instead (the links above, you can also produce statically linkable binaries but it might have issues on e.g. macOS which has...not the most reliable linker that likes to take breaking changes). This type of library works in almost every scenario a library implemented in C/C++/Rust with C exports does. For example, here someone implemented a hello-world demonstration of using C# to write an OBS plugin: https://sharovarskyi.com/blog/posts/dotnet-obs-plugin-with-nativeaot/ https://sharovarskyi.com/blog/posts/dotnet-obs-plugin-with-n... Using the exports boils down to just this https://github.com/kostya9/DotnetObsPluginWithNativeAOT/blob/main/ObsPlugin.cs https://github.com/kostya9/DotnetObsPluginWithNativeAOT/blob... and specifying correct build flags.
  - duped 3y ago
    I haven't been looking for those because I don't work with .NET. Regardless, what you're linking still needs callers and callees to agree on calling convention and special binding annotations across FFI boundaries which isn't particularly interesting from the perspective of language implementation like the promises of Graal or WASM + GC + component model.
    - neonsunset 3y ago
      There is no free lunch. WASM just means another lowest common denominator abstraction for FFI. I'm also looking forward to WASM getting actually good so .NET could properly target it (because shipping WASM-compiled GC is really, really painful, it works acceptably today, but could be better). Current WasmGC spec is pretty much unusable by any language that has non-primitive GC implementation. Just please don't run WASM on the server, we're already getting diminishing generational performance gains in hardware, no need to reduce them further. The exports in the examples follow C ABI with respective OS/ISA-specific calling convention.
duped 3y ago
There are more managed langauges than Go, Java, and C#. Swift (and Objective C with ARC) are a bit different in that they don't use mark and sweep/generational GCs for automatic memory management so it's significantly less of an issue. Compare with Lua, Python, JS, etc where there's a serialization boundary between the two. But I stand by what I said. It's generally unwise to mix the two, particularly calling unmanaged code from managed code. I wouldn't say it's "not a problem" because there are very few environments where you don't pay some cost for mixing and matching between managed/unmanaged code, and the environments designed around it are built from first principles to support it, like .NET. More interesting to me are Graal and WASM (once GC support lands) which should make it much easier to deal with.
meindnoch 3y ago
Swift is not a "managed" (i.e. GC) language.
- pjmlp 3y ago
  Reference counting is a GC algorithm in any decent CS book. "A Unified Theory of Garbage Collection" https://courses.cs.washington.edu/courses/cse590p/05au/p50-bacon.pdf https://courses.cs.washington.edu/courses/cse590p/05au/p50-b...
  - meindnoch 3y ago
    I was expecting this pedantic comment... If refcounting makes a language "managed", then C++ with shared_ptr is also "managed". _______ The charitable interpretation is that OP was likely referring to the issues when calling into a language with a relocating GC (because you need to tell the GC not to move objects while you're working with them), which Swift is not.
    - neonsunset 3y ago
      Swift has just as much concerns for its structs and classes passing across FFI in terms of marshalling/unmarshalling and ensuring the ARC-unaware code performs either manual retain/release calls or adapts them to whatever other mechanism of memory management of the callee. One of the comments here mentions that Swift has its own stable ABI, which exposes richer type system, so it does stand out in terms of interop (.NET 9 will add support for it natively (library evolution ABI) without having to go through C calls or C "glue" code on iOS and macOS, maybe the first non-Swift/ObjC platform to do so?). Object pinning in .NET is only a part of equation and at this point far from the biggest concern (it just works, like it did 15 years ago, maybe it's a matter of fuss elsewhere?).
    - pjmlp 3y ago
      Nope, because that is a library class without any language support. The pedantic comment is a synonymous with proper education instead of street urban myths.
pjmlp 3y ago
Except that is only true since those attributes were introduced in recent .NET versions, and it doesn't account for COM marshaling issues. Plenty of .NET code still using the old ways that isn't going to be rewritten, either for these attributes, or the new Cs/WinRT, or the new Core COM interop, which doesn't support all COM use cases anyway.
- neonsunset 3y ago
  Code written for .NET Framework is completely irrelevant to conversation since it does not evaluate it. You should treat it as dead and move on because it does not impact what .NET can or can’t do. There is no point to bring up “No, but 10 years ago it was different”. So what? It’s not 2014 anymore.
  - pjmlp 3y ago
    My remarks also apply to modern .NET, as those improvements were introduced in .NET 6 and .NET 8, and require a code rewrite to adopt them, instead of the old ways which are also available, in your blind advocacy you happened to miss out. Very few code gets written from scratch unless we are talking about startups.

zozbot234 3y ago

> It's usually unwise to mix managed and unmanaged memory Broadly stated, you can achieve this by marking a managed object as a GC root whenever it's to be referenced by unmanaged code (so that it won't be freed or moved in that case) and adding finalizers whenever managed objects own or hold refcounted references over unmanaged memory (so that the unmanaged code can reason about these objects being freed). But yes, it's a bit fiddly.