5 ms·

I am curious if this is actually using the official .NET6+ runtime and how gen2 collections are managed. GC can become a monster problem once you start scaling

I am curious if this is actually using the official .NET6+ runtime and how gen2 collections are managed. GC can become a monster problem once you start scaling up the # of things in a system like this.

szundi 4y ago
Exactly, gc pauses kind of bugging me in a lot of games based on .net, like Kerbal
- kevingadd 4y ago
  Generally speaking if a .NET game has GC pauses, it's probably because they didn't try. You can represent most game state using structs and arrays without significant ergonomics problems, and your GCs for the classes you do heap allocate will be much shorter if you design your data structures to be quick to trace. Things were different for Unity games using the Boehm collector though, since it's a conservative non-generational collector that as a result is very slow to collect and collects more frequently. That's why you saw a lot of frequent pauses in Unity games (my understanding is they no longer use Boehm). The game I'm currently working on GCs maybe 1-2 times per minute and the pauses are short enough that it doesn't drop frames, despite the fact that it frequently allocates strings and uses 'async Task' extensively. If you tune the collector settings you can control this even more, and you also have the option to manually force GCs to happen while the game is waiting for vsync, which hides them further. Visual Studio has a built in allocation profiler that can help if you're looking to optimize your games for fewer GCs, and to make tracing faster what you want to do is avoid having any reference types in your struct arrays (use handles instead), which will let the GC skip tracing your big arrays of structs entirely. So for example the big tables of draw calls I use refer to textures by numeric handle instead of reference, which means I can have a 50mb table without making GCs slower. Using handles instead of references also means that copying the struct doesn't require write barriers, which will boost your overall perf a bit.
- pjmlp 4y ago
  Not all .NET games are made equal. First of all, there is the runtime they are actually using. Then the language version. Finally, how well the devs know how to do low level optimisations.
  - omnicognate 4y ago
    Unfortunately you mostly don't do low-level optimisations in C#. You can manipulate raw memory but unsafe C# is horrifyingly unsafe - far more dangerous than writing C - and code written with it has to navigate a minefield to interact safely with ordinary C#. Instead, heavy optimisation in C# is usually "high level but knowing the runtime in great detail" optimisation. Stuff like knowing the hairy details of the GC implementation (eg by reading Pro .NET Memory Management and/or the single 30,000 line C++ file that contains the GC implementation [1]) and using this knowledge to write ordinary C# code in such a way that the GC can handle its usage patterns very efficiently, or doing things like object pooling that try to cut the GC out of the picture altogether. Essentially, it's knowing what you want the machine to do and figuring out a way to trick the runtime into doing that instead of what it wants to do. I find it massively frustrating and after doing it for a few years I moved back to languages that try to do what you tell them to do instead of the other way round. Some people seem to thrive on it, though, and manage to get impressive results - see, eg. Marc Gravell's blog. [1] https://github.com/dotnet/runtime/blob/main/src/coreclr/gc/gc.cpp?raw=true https://github.com/dotnet/runtime/blob/main/src/coreclr/gc/g...
    - bob1029 4y ago
      I agree that understanding how the GC was built is very important. Little nuances like forcing workstation mode and explicitly calling GC.Collect can make all the difference. You can get very far with a GC language like this. The part that bothers me are the built-ins that allocate. TPL, AspNetCore, et. al. If I want a truly zero-alloc C# solution, I have to go all the way to the bottom with abstractions like Socket, NetworkStream, SslStream, etc.
    - pjmlp 4y ago
      I advise to update yourself what using C#11 offers in possibilities beyond only unsafe code blocks. Based on my C and C++ years, it is hardly any difference from doing optimization, knowing what the compiler actually generates, how is the runtime and standard library actually implemented, and above all getting to use the proper data structures, algorithms and a profiler.
tebjan 4y ago
Stride is written in C#, uses the official .NET6 and was developed by world-class engineers. It takes this topic very serious. The game loop has almost zero allocations. It's a very good learning source for writing high performance C# for real-time applications. As a Stride user you have to follow similar guide lines in your scripts to avoid pressure on the garbage collector. But that's not so difficult as soon as you get the hang of it.
alkonaut 4y ago
Not sure how this works under the hood but if I made a game in .NET I'd basically have a ban on any per-frame allocations. That means you have to be quite non-idiomatic when it comes so C# within the game loop, e.g. no dynamic collections (resizing will allocate), no "foreach" risks allocating enumerators. There are very few "zerp cost abstractions" in C# so when you want zero cost, you must also avoid some abstractions. Basically you use a very C-looking subset of C# for the parts of the code that run in the game loop. Allocate everything up front, use pools and arenas, etc. etc. It's easy to trip up but tooling is good so it's easy to notice when you accidentally introduced allocation. The engine isn't the problem, it's that when you expose C# as your scripting language then users might think they can write "normal" C# in their game scripts. Which you probably don't want to do.
- bob1029 4y ago
  > I'd basically have a ban on any per-frame allocations. I have a prototype where I intentionally invoke GC after every frame. My reasoning is that some amount of GC will eventually need to occur, so why not do it as frequently as possible and at the exact moment where we can compensate with other measures, such as the frame scheduler & time step abstractions. Even with heavy allocation (i.e. deserializing the scene graph from a wire protocol before every frame), this approach seems to keep GC pauses bounded to ~1-5 milliseconds in my implementations. If I did not explicitly call GC when I thought best, the runtime would decide to run a 100-1000 ms collection at its convenience and the entire experience would become worthless to human perception.
  - kevingadd 4y ago
    This is a great strategy as long as you can keep the pause times low enough - especially if you manage to hide the gc behind vertical sync. Getting the exact timing for that is a little tricky, but if your game is running fast enough you've probably got upwards of 5ms waiting to be used while your main thread blocks inside Present/SwapBuffers. (Because those functions are native OS APIs, there won't be managed code on the stack and the GC should be able to run.)
  - starburst 4y ago
Kukumber 4y ago
not just the GC, but also the JIT is a problem .NET platform is server focused, it does a lot at runtime, including recompiling bunch of functions It was a huge problem before Unity had the IL2CPP target, you had to deal with the slow cold start and you also had to manually call your hot functions to avoid getting them recompiled during gameplay, wich would cause ton of micro-stutters Hence people prefer languages like LUA for their scripting needs, even for Unity [1], great perf for interpreted and you can ship it on consoles/mobile platform that forbid JIT Now that .net has a dedicated AOT compiler, things could improve, but i doubt this engine supports NativeAOT [1] - https://github.com/Tencent/xLua https://github.com/Tencent/xLua
ykafia 4y ago
Official runtime yes. For having worked with the source, I can safely say it has been written to not stress the GC too much. But it's a very modular engine, you can change parts of it to better accommodate your needs, I'm currently finishing implementing my "no-allocation" ECS library and will 100% try to integrate it into my fork.

szundi 4y ago

Exactly, gc pauses kind of bugging me in a lot of games based on .net, like Kerbal

kevingadd 4y ago
Generally speaking if a .NET game has GC pauses, it's probably because they didn't try. You can represent most game state using structs and arrays without significant ergonomics problems, and your GCs for the classes you do heap allocate will be much shorter if you design your data structures to be quick to trace. Things were different for Unity games using the Boehm collector though, since it's a conservative non-generational collector that as a result is very slow to collect and collects more frequently. That's why you saw a lot of frequent pauses in Unity games (my understanding is they no longer use Boehm). The game I'm currently working on GCs maybe 1-2 times per minute and the pauses are short enough that it doesn't drop frames, despite the fact that it frequently allocates strings and uses 'async Task' extensively. If you tune the collector settings you can control this even more, and you also have the option to manually force GCs to happen while the game is waiting for vsync, which hides them further. Visual Studio has a built in allocation profiler that can help if you're looking to optimize your games for fewer GCs, and to make tracing faster what you want to do is avoid having any reference types in your struct arrays (use handles instead), which will let the GC skip tracing your big arrays of structs entirely. So for example the big tables of draw calls I use refer to textures by numeric handle instead of reference, which means I can have a 50mb table without making GCs slower. Using handles instead of references also means that copying the struct doesn't require write barriers, which will boost your overall perf a bit.
pjmlp 4y ago
Not all .NET games are made equal. First of all, there is the runtime they are actually using. Then the language version. Finally, how well the devs know how to do low level optimisations.
- omnicognate 4y ago
  Unfortunately you mostly don't do low-level optimisations in C#. You can manipulate raw memory but unsafe C# is horrifyingly unsafe - far more dangerous than writing C - and code written with it has to navigate a minefield to interact safely with ordinary C#. Instead, heavy optimisation in C# is usually "high level but knowing the runtime in great detail" optimisation. Stuff like knowing the hairy details of the GC implementation (eg by reading Pro .NET Memory Management and/or the single 30,000 line C++ file that contains the GC implementation [1]) and using this knowledge to write ordinary C# code in such a way that the GC can handle its usage patterns very efficiently, or doing things like object pooling that try to cut the GC out of the picture altogether. Essentially, it's knowing what you want the machine to do and figuring out a way to trick the runtime into doing that instead of what it wants to do. I find it massively frustrating and after doing it for a few years I moved back to languages that try to do what you tell them to do instead of the other way round. Some people seem to thrive on it, though, and manage to get impressive results - see, eg. Marc Gravell's blog. [1] https://github.com/dotnet/runtime/blob/main/src/coreclr/gc/gc.cpp?raw=true https://github.com/dotnet/runtime/blob/main/src/coreclr/gc/g...
  - bob1029 4y ago
    I agree that understanding how the GC was built is very important. Little nuances like forcing workstation mode and explicitly calling GC.Collect can make all the difference. You can get very far with a GC language like this. The part that bothers me are the built-ins that allocate. TPL, AspNetCore, et. al. If I want a truly zero-alloc C# solution, I have to go all the way to the bottom with abstractions like Socket, NetworkStream, SslStream, etc.
  - pjmlp 4y ago
    I advise to update yourself what using C#11 offers in possibilities beyond only unsafe code blocks. Based on my C and C++ years, it is hardly any difference from doing optimization, knowing what the compiler actually generates, how is the runtime and standard library actually implemented, and above all getting to use the proper data structures, algorithms and a profiler.

tebjan 4y ago

Stride is written in C#, uses the official .NET6 and was developed by world-class engineers. It takes this topic very serious. The game loop has almost zero allocations. It's a very good learning source for writing high performance C# for real-time applications. As a Stride user you have to follow similar guide lines in your scripts to avoid pressure on the garbage collector. But that's not so difficult as soon as you get the hang of it.

alkonaut 4y ago

Not sure how this works under the hood but if I made a game in .NET I'd basically have a ban on any per-frame allocations. That means you have to be quite non-idiomatic when it comes so C# within the game loop, e.g. no dynamic collections (resizing will allocate), no "foreach" risks allocating enumerators. There are very few "zerp cost abstractions" in C# so when you want zero cost, you must also avoid some abstractions. Basically you use a very C-looking subset of C# for the parts of the code that run in the game loop. Allocate everything up front, use pools and arenas, etc. etc. It's easy to trip up but tooling is good so it's easy to notice when you accidentally introduced allocation. The engine isn't the problem, it's that when you expose C# as your scripting language then users might think they can write "normal" C# in their game scripts. Which you probably don't want to do.

bob1029 4y ago
> I'd basically have a ban on any per-frame allocations. I have a prototype where I intentionally invoke GC after every frame. My reasoning is that some amount of GC will eventually need to occur, so why not do it as frequently as possible and at the exact moment where we can compensate with other measures, such as the frame scheduler & time step abstractions. Even with heavy allocation (i.e. deserializing the scene graph from a wire protocol before every frame), this approach seems to keep GC pauses bounded to ~1-5 milliseconds in my implementations. If I did not explicitly call GC when I thought best, the runtime would decide to run a 100-1000 ms collection at its convenience and the entire experience would become worthless to human perception.
- kevingadd 4y ago
  This is a great strategy as long as you can keep the pause times low enough - especially if you manage to hide the gc behind vertical sync. Getting the exact timing for that is a little tricky, but if your game is running fast enough you've probably got upwards of 5ms waiting to be used while your main thread blocks inside Present/SwapBuffers. (Because those functions are native OS APIs, there won't be managed code on the stack and the GC should be able to run.)
- starburst 4y ago

Kukumber 4y ago

not just the GC, but also the JIT is a problem .NET platform is server focused, it does a lot at runtime, including recompiling bunch of functions It was a huge problem before Unity had the IL2CPP target, you had to deal with the slow cold start and you also had to manually call your hot functions to avoid getting them recompiled during gameplay, wich would cause ton of micro-stutters Hence people prefer languages like LUA for their scripting needs, even for Unity [1], great perf for interpreted and you can ship it on consoles/mobile platform that forbid JIT Now that .net has a dedicated AOT compiler, things could improve, but i doubt this engine supports NativeAOT [1] - https://github.com/Tencent/xLua https://github.com/Tencent/xLua

ykafia 4y ago

Official runtime yes. For having worked with the source, I can safely say it has been written to not stress the GC too much. But it's a very modular engine, you can change parts of it to better accommodate your needs, I'm currently finishing implementing my "no-allocation" ECS library and will 100% try to integrate it into my fork.