5 ms·
Writing a .NET Garbage Collector in C# – Part 1
- etra0 2y agoI'm very much looking up to NativeAOT on C#, being able to compile to native dlls is very attractive to me, as I'd love to do (offline) game modding. I currently use Rust and it works quite well, but it's never too late to wish for a more 'forgiving' language for smaller projects!
- MrLeap 2y agoInteresting. You and I are reasonably adjacent it seems like. I've been cooking up a pleasant way to enable modding support in games I make. I have this idea in my head that I want this system to work even all the way up to letting people host their own servers that run their own mods without allowing them the power to do nefarious things to the clients who connect. This means a scripting language layer, most commonly LUA. For a bunch of reasons, many are vibes based, I've decided to go with a lisp instead. After doing a bit of research I found this repo to use as a starting point. https://github.com/microsoft/schemy https://github.com/microsoft/schemy Checks all my boxes. Big ones being no new dependencies and not many lines of code. It took just a few minutes to get a handle on the whole thing end-to-end. I'm working right now on setting a little reflection metaprogramming that would expose any functions that I put an attribute above to the lisp layer. There's a few things to delete so it's safe to serialize over the wire, but it looks like it'll enable what I'm after. I hope I can trust my serializers :D What kinds of mods do you want to make?
- etra0 2y agoI'm interested in something that we call "Virtual Photography", which is basically capture beauty within virtual worlds. For that, there are games that include photo modes and some that don't, but it's often that those they do have several limitations (like range, controls, etc). I mostly do free-cameras, a way to detach the camera from the player to allow different angles that otherwise would be impossible to appreciate. Lately I did a project to spawn lights on The Witcher 3 [1], to allow 'virtual photographers' to take some amazing portraits [2, 3], I did this by using Rust with a little bit of x86 assembly. Rust has been neat for this, because despite working on a safety-hazardous territory, the amount of crashes has been minimal because we still have safeguards within the unsafe stuff, but sometimes I just want to not worry about lifetimes. If you like games, you can check this amazing gallery from the FRAMED community to see the extents the people do to take amazing shots within games engines (and admittedly, external tools like reshade, mods, etc): https://framedsc.com/HallOfFramed https://framedsc.com/HallOfFramed [1] https://github.com/etra0/litcher https://github.com/etra0/litcher [2] https://framedsc.com/HallOfFramed/?game=The+Witcher+3&title=The+Witcher+3&imageId=1674303289 https://framedsc.com/HallOfFramed/?game=The+Witcher+3&title=... [3] https://framedsc.com/HallOfFramed/?game=The+Witcher+3&title=The+Witcher+3&imageId=1673208744 https://framedsc.com/HallOfFramed/?game=The+Witcher+3&title=...
- MrLeap 2y agoThis is absolutely incredible work! How hard was it to get your foot in the door on having code execute on an already running process? Do you do this from a separate process at runtime or is it all modifying the executable before you run it? I've never done anything like that. I can imagine getting the memory offsets for things like camera transforms being reasonably straight forward with cheat engine, but mutating the values feels a like converting a train from diesel to electric while it's carrying a load. It reminds me of something Ross Scott (he makes youtube videos about games) has expressed a desire to have -- the ability to comprehensively map and record the 3d worlds of games that are going to get shut down. Admittedly I initially dismissed it a little on the grounds that if you want that, just extract the 3d model data from the client and load up blender. Seeing what you and others have done is making me appreciate what he's after more. The whole composition, environments and characters and lighting and visual effects and post processing would not be preserved just by yanking out the .fbx files and textures. A lot would be lost. Despite being a gamedev myself, seems like I somehow lost the perspective a little. It's awesome to see there's a community and a hobby doing what you're doing. Thanks for showing this to me.
- etra0 2y ago> This is absolutely incredible work! Thanks for your kind words! > How hard was it to get your foot in the door on having code execute on an already running process? Do you do this from a separate process at runtime or is it all modifying the executable before you run it? All is done at runtime. By nature of how Windows DLL work, they have a DllMain that's executed every time you load a dynamic library into a running process. From there you spawn a thread and you can do pretty much whatever you want within the same memory space of the games. There are multiple ways of injecting a DLL into a running process, and since I only care about offline games, I don't have to fight against any anticheat and stuff so it is pretty straightforward, it is a very interesting topic to read about to be honest! > I can imagine getting the memory offsets for things like camera transforms being reasonably straight forward with cheat engine Cheat Engine is an amazing tool, for real. I remember using it when I was a kid to do very rudimentary cheating -- just scanning and changing values; years have passed and one day I opened Cheat Engine again to... skip over an annoying mechanic of a game and I noticed it can do so much more! it has a very good debugger, disassembler, memory view, you can even inject your own assembly code. It was the gateway drug that I needed to get started on everything I do. > but mutating the values feels a like converting a train from diesel to electric while it's carrying a load Well, compilers are pretty straightforward (at least, most of the time), it's all about memory layout. If you have something like struct vec { float x; float y; float z; }; Once you find the pointer to the objects you need, most certainly they're going to be each one after the other. This looks like obvious knowledge, but at least on my brain I had to build an extra bridge to realize how simple it is :) (of course things can always get complicated but this is the basic gist of it). And as long as you know your programming language has a stable memory layout, you can map it from your code. I even was able to map some very basic virtual class from C++ within Rust by mapping the virtual tables. What I enjoy the most, is that I learned a lot about low-level programming, assembly, how OS and compilers work together, while having fun playing games, and developing tools that people can use to create pretty images -- it felt like the outcome was extremely positive and it brings me a lot of joy.
- Dwedit 2y agoI think that .NET DLLs can still be loaded by native code? I could throw together a project to check if that actually works or not. Edit: Using the tool "dllexport" (https://github.com/3F/DllExport https://github.com/3F/DllExport), you can create a .NET DLL that exports a function that is callable from C/C++ code. Just LoadLibrary and GetProcAddress.
- etra0 2y agooh, neat. I recently needed to export some functions from a C# project to be used by C++ code, but at the time I researched it (June 2024), the dllexport project seemed to be frozen (last commit before that date was on 2021). Thanks to NativeAOT, though, it seems that you can do this kind of stuff without extra dependencies which is great too!
- rpeden 2y agoIt works quite well especially in .NET 9! And exporting to WASM works nearly identically to DllExport. I used that to shift all the heavy lifting to .NET 9 AOT compiled to WASM in a fun little side project I've been working on: https://evo.ryanpeden.com https://evo.ryanpeden.com
- rasmuskl 2y agoCan you share more on this? Any examples that helped you or anything you've written on the WASM .NET topic?
- rpeden 2y agoNot yet - I just created a browser-wasm project using the .NET CLI and then experimented with it. I spent a bunch of the digging through .targets files to see what optimization options were available. I plan to put the source on GitHub shortly so others can use it as an example. Just need to clean things up a little first.
- 2y ago
- DeathArrow 2y agoI am curious if there is a way to discard GC and free the memory by hand. This article might point me in the right direction to disable GC, but freeing memory can't be as simple.
- Dwedit 2y agoIf you really really want to, you can use Marshal.AllocHGlobal and allocate unmanaged memory. All access happens through pointers and `unsafe` code blocks.
- neonsunset 2y agoYou're supposed to use NativeMemory.Alloc/.Free methods instead. Obviously nothing prevents you from PInvoking malloc and free directly, or linking to a custom allocator, or even using https://www.nuget.org/packages/TerraFX.Interop.Mimalloc https://www.nuget.org/packages/TerraFX.Interop.Mimalloc which is a fully managed Mimalloc implementation (which is competitive with the original). There is also an advanced undocumented API to register a specific segment of memory as nongc-heap where you can allocate otherwise managed objects manually. But the most important thing to understand that normally in languages with generational GCs the objects are not freed, instead, the surviving objects are copied to an older generation while the rest of the memory is reclaimed and immediately made available for subsequent allocations. As a result, you can't "free" an object since such operation does not exist in the design. Overall, you're not forced to use objects in low-level code - structs can implement methods, interfaces, used in generics for zero cost abstractions,etc., and if you don't allocate or allocate only a little - the GC will never run.
- algorithmsRcool 2y agoI didn't realize that the GC copied the surviving objects to a separate location. Come to think of it, i don't actually know how the GC keeps track of what object lies in what generation.
- mystified5016 2y ago
- amne 2y agoThis right here is peak Microsoft: set DOTNET_GCName=clrgc.dll set COMPlus_GCPath=ManagedDotnetGC.dll Good luck understanding this when you come back 2 months later.
- CharlieDigital 2y agoJust write a batch file? I don't see the issue
- mystified5016 2y agoExplain what these lines do and why they're necessary, then you'll understand the problem
- CharlieDigital 2y agoThey are environment variables specifying DLLs to use.
- MortyWaves 2y agoIt was my understanding the ridiculous and historic COMPlus name had been replaced with just DOTNET several years ago.
- pjc50 2y agoDeeper horror: > I got closer to the goal when I realized an interesting quirk: .NET supports environment variables prefixed by either DOTNET_ or COMPlus_, whereas NativeAOT only supports DOTNET_. So if we set COMPlus_GCName=ManagedDotnetGC.dll, only the .NET runtime will pick it up, and the NativeAOT runtime will ignore it.
- pjmlp 2y agoIt is rather hard to name environment variables as COM+.
- 2y ago
- partomniscient 2y agoFor anyone else who's now out of touch with this stuff as me: AOT - Ahead Of Time. Performance improvements looked really impressive until I later saw it was only partially supported: https://learn.microsoft.com/en-us/aspnet/core/fundamentals/native-aot?view=aspnetcore-9.0 https://learn.microsoft.com/en-us/aspnet/core/fundamentals/n...