8 ms·

> For a video game running at 60 frames per second each frame has 16 milliseconds in which to complete, so a pause of less than a millisecond is not going to be

> For a video game running at 60 frames per second each frame has 16 milliseconds in which to complete, so a pause of less than a millisecond is not going to be significant relative to other overheads. This is definitely very significant, you just took away 6% of the CPU time available to the game to calculate everything necessary for that frame. And that's if there's only one pause inside each frame. And if you are running close to 100% CPU time, and the pause happens right before the vsync, you might miss the vsync and you will drop a whole frame. This is not only a problem with GC, even with more deterministic memory management, like plain malloc() and free() in C, you can have pauses if the operating system needs to go and find some free memory, update page tables, flush TLB caches and so on. So for games and anything real-time, you probably want to avoid memory allocations as much as possible, and certainly don't want your whole process being paused at random. So if you are stuck with GC, then one question would be: can GC be done for individual threads without blocking other threads?

deleted 7y ago
[deleted]
dkersten 7y ago
Yeah, there's a reason games often preallocate large chunks of memory and then not allocate/deallocate during a level, use object pooling and stack allocators. It seems like a strange example to give, when many modern games obsess over minimising delays in all sorts of ways (eg anything Mike Acton talks about: not just cache-friendly data-oriented structure-of-arrays or removing branches, but also eliminating function calls ("where there's one, there's many") and pointer indirection).
- masklinn 7y ago
  > It seems like a strange example to give, when many modern games obsess over minimising delays in all sorts of ways And the more competitive ones tend to run higher than 60fps too, making the issue even more prominent, meanwhile for those for which it doesn't really matter (e.g. a 2D adventure game)… neither does ultra-low GC latency either.
w0utert 7y ago
True. This is why any serious game running on a VM with GC would avoid any kind of per-frame memory allocation and use arena allocators instead. The whole premise of a GC to save you from thinking about memory allocation is moot if it comes to real-time sensitive tasks, there it makes things harder instead of easier. A reference counting system is much better IMO, even though you may be able to get higher absolute throughput with GC, at least the small overhead you incur is stable and predictable.
- masklinn 7y ago
  > True. This is why any serious game running on a VM with GC would avoid any kind of per-frame memory allocation and use arena allocators instead. The problem is having the VM allow and support that usage, and the JVM definitely is not best-of-class there. See ixy[0] from a few months back where the researchers / developers didn't manage to get under > ~20 bytes of allocation per forwarded packet in Java. [0] https://github.com/ixy-languages/ixy-languages https://github.com/ixy-languages/ixy-languages
- _ph_ 7y ago
  I don't think the idea of GC is to save you from thinking about memory allocation. Actually, not thinking about memory allocation is a great way to create very slow programs, independant of whether you us an GC or manually free memory, though modern GCs keep the performance penalty reasonably low. Still, you want to think about memory allocation, so that you don't allocate inefficiently. What GC delivers is first of all the guarantee of correctness, a pointer is either nil or points to a validly allocated memory. Also, it removes most needs of bookkeeping. That simplifies programs and especially allows to write clean APIs, where functions are fine to allocate reasonable amounts of memory. But wherever there are large parts of memory required, especially with a clear lifetime, I think a lot about memory allocation and even in a GC language I try to reuse memory wherever that makes sense. So for a game engine, there should not be much need for allocation during the run time of a "level" and thus GC pauses should only happen between levels.
  - barrkel 7y ago
    Not thinking about ownership over references to heap allocations is very very much the big productivity win from GC. Memory safety is usually an additional bonus, but it's a distinct thing; it's possible to have (conservative or refcounting) GC without memory safety and vice versa.
brighteyes 7y ago
You're right, yes - a 1ms pause has a cost, even in a 16ms budget. I interpreted the author's point as "GC is now potentially viable for games", in the sense that older GCs might have pause times larger than 16ms, making them obviously inappropriate, whereas today 1ms pauses is something that can be budgeted for at least in some cases. As you said, malloc/free also have costs, and also GC has other benefits, like bump allocation, moving things to compact memory and improve cache locality, etc. 1ms pauses means GC is worth considering even for a game, in other words.
- walkingolof 7y ago
  "Games" is a term for are a large collection of very different types of software, do we mean AAA FPS games? Then I would probably not use a GC, if we mean most other games? Then it's just fine. Many GC events are way shorter than 1ms also...
  - shrimp_emoji 7y ago
    Turn-based or pausable-real time 4X games where you spend most of your time paused and also get hit by a regular (~1000 ms) autosave freeze spring to mind.
  - dfox 7y ago
    If you look around configuration REDEngine configuration files in PC version of Witcher 3, there is file with what for me looks like tuning parameters of Boehm-like conservative tracing GC, it is probably the only file in there that even includes comments about what the parameters mean and partially how they were determined. My overall impression from the various configuration files is that the whole world streaming/demand-paging mechanism is somehow interwoven with or even partially driven by the GC (so the GC is not meant for some small-ish scripting heap)
- arcticbull 7y ago
  > As you said, malloc/free also have costs, and also GC has other benefits, like bump allocation, moving things to compact memory and improve cache locality, etc. 1ms pauses means GC is worth considering even for a game, in other words. The general approach used by games is probably superior, which are arenas or zones. One approach games use is to allocate everything that's necessary for a given frame in a single large block, carve it off over time as needed, then drop the entire arena. All allocations then have memory locality, no fragmentation, basically zero cost to allocate (increment a pointer) and zero cost to deallocate. This kind of thing actually plays really nicely with Rust's lifetimes since you can couple frame lifetime with objects in that frame and get static validation. Arenas are already available in nightly [1]. [1] https://doc.rust-lang.org/nightly/nightly-rustc/arena/index.html https://doc.rust-lang.org/nightly/nightly-rustc/arena/index....
cmrdporcupine 7y ago
The other question is: can I at least time this GC so it doesn't overlap the refresh interval and skip a frame? Because 1ms may not be significant if it lands in the middle of the 16ms period, but if it lands at the end you've just produced jitter. So I'd think a GC designed for a gaming system would take this into account; incremental concurrent GC that can be timed to happen at the start of each each frame interval and be paused if it gets close to the end.
- corysama 7y ago
  You can manually trigger GC in Lua. I’ve used this to trigger light GC work every frame in some games to buy more consistent GC times (no spikes) at the cost of higher average times.
  - brighteyes 7y ago
    Chrome does this too, it times GCs for when it has free time or is rendering in another thread, https://blog.chromium.org/2015/12/smarter-garbage-collection-for-smoother.html https://blog.chromium.org/2015/12/smarter-garbage-collection...
- markdog12 7y ago
  I remember the oddly named System.pauseForGCIfCollectionImminent() AS3 (Flash) function: https://help.adobe.com/en_US/FlashPlatform/reference/actionscript/3/flash/system/System.html#pauseForGCIfCollectionImminent() https://help.adobe.com/en_US/FlashPlatform/reference/actions...
skybrian 7y ago
Would it make a difference for Minecraft? I haven't played in a while but I remember fiddling with JVM settings to try to get smoother performance.
- pjmlp 7y ago
  I lost the link to the forum post, where it was described how bad Minecraft data structures were designed.
  - blattimwind 7y ago
    Many years ago, when I used to play Minecraft, the common wisdom was simply to get as much memory as possible, so that the GC runs less frequently. With the debug overlay you could see how the game kept allocating at about a rate of 300-500 MB/s until the JVM GC was triggered, which created a noticeable lag in the game. I used to know a few MC server admins, and I was surprised just how massive the resource consumption of the game was. One server I usually played on ran on multiple machines. Specs were I believe around 128 GB memory, 32 cores, 16 or so dedicated to GC...
    - imtringued 7y ago
      Minecraft world simulation runs on a single core so you won't see much benefit from more cores. GC may be an exception though.
    - pjmlp 7y ago
      Something I remember from that forum post was that instead of using array based representation for the meshes, it uses classes for every kind of information (Vector, Points,...). Very nice school OOP, but not always the best way.
badsectoracula 7y ago
Considering how variable game frames are, even if the GC eats that 1ms it will be unnoticeable, assuming it doesn't happen too often. I mean, i play games like Skyrim that now and then give you a pause as you walk the world while it unloads cells (i refer to normal walking around, not entering/exiting somewhere - which, btw, would be a good place to force a GC too in a similar game that would be written in a GC'd language) which can even be 1 second long. 1ms is noise long lost in there.
- blattimwind 7y ago
  > Considering how variable game frames are, even if the GC eats that 1ms it will be unnoticeable, assuming it doesn't happen too often. For a competitive game you are looking at a refresh of 240 Hz (4 ms) but people generally target frame rates of at least 300-400 Hz (2.5 ms per frame!). There is simply no room to randomly throw in 1 ms pauses here and there without introducing micro-stuttering. Even for a casual game 1 ms of extra frame time variance is bad. If your Skyrim has one second freezes while roaming the world, your install is busted, because the game in proper working order doesn't do that.
  - badsectoracula 7y ago
    > For a competitive game you are looking at a refresh of 240 Hz Yes, it depends on the game, what i wrote doesn't apply to every single game ever made out there, but it does apply to a ton of games. > Even for a casual game 1 ms of extra frame time variance is bad. If it happens every several minutes you will 100% not notice it at all. In pretty much every high end 3D game (especially an open world one) you'll get way more variance by turning the camera around and/or just walking than that. > If your Skyrim has one second freezes while roaming the world, your install is busted, because the game in proper working order doesn't do that. I'm 100% sure my Skyrim install is perfectly fine (it is a fresh one) and it does happen, just not frequently. If you haven't noticed it... well, as i already wrote, it isn't really noticeable. And - especially on Skyrim - you'll also get way more variance than 1ms just walking around.
    - 7y ago
pron 7y ago
Games have a very regular object lifetime, allowing them to manage much of their allocations in arenas, which can certainly beat even modern GCs in performance (although maybe not for good). But concurrent server business applications don't have such regular object lifetimes, and it is getting quite hard to beat the performance of modern GCs with manual memory management in such applications. These days, footprint is the major and pretty much only significant cost to GCs in a great many applications.
hinkley 7y ago
My second job was displaying telemetry waveforms in real time, in Java... 1.3 if memory serves. MVP goal was 4 waveforms, I managed to get 16. What we did to accomplish this was first aggressively optimize the data ingestion, then spend all that surplus on maintaining a ring buffer of pixels. This meant we were always a little behind live (.5s or 1s, I can't recall), but it made everything smooth as silk. To maintain homeostasis, I'd trigger GC every time we made it through a paint cycle with a full buffer. There are ways to do it, but it could change the structure of your app. I say, "in our case the main paint loop was stupid-simple so it wasn't that big of a deal to tack it on," but the idea was always sort of in the back of my mind so it definitely informed the design. What I didn't have to deal with is random inputs from the user changing my display strategy.

deleted 7y ago

[deleted]

dkersten 7y ago

Yeah, there's a reason games often preallocate large chunks of memory and then not allocate/deallocate during a level, use object pooling and stack allocators. It seems like a strange example to give, when many modern games obsess over minimising delays in all sorts of ways (eg anything Mike Acton talks about: not just cache-friendly data-oriented structure-of-arrays or removing branches, but also eliminating function calls ("where there's one, there's many") and pointer indirection).

masklinn 7y ago
> It seems like a strange example to give, when many modern games obsess over minimising delays in all sorts of ways And the more competitive ones tend to run higher than 60fps too, making the issue even more prominent, meanwhile for those for which it doesn't really matter (e.g. a 2D adventure game)… neither does ultra-low GC latency either.

w0utert 7y ago

True. This is why any serious game running on a VM with GC would avoid any kind of per-frame memory allocation and use arena allocators instead. The whole premise of a GC to save you from thinking about memory allocation is moot if it comes to real-time sensitive tasks, there it makes things harder instead of easier. A reference counting system is much better IMO, even though you may be able to get higher absolute throughput with GC, at least the small overhead you incur is stable and predictable.

masklinn 7y ago
> True. This is why any serious game running on a VM with GC would avoid any kind of per-frame memory allocation and use arena allocators instead. The problem is having the VM allow and support that usage, and the JVM definitely is not best-of-class there. See ixy[0] from a few months back where the researchers / developers didn't manage to get under > ~20 bytes of allocation per forwarded packet in Java. [0] https://github.com/ixy-languages/ixy-languages https://github.com/ixy-languages/ixy-languages
_ph_ 7y ago
I don't think the idea of GC is to save you from thinking about memory allocation. Actually, not thinking about memory allocation is a great way to create very slow programs, independant of whether you us an GC or manually free memory, though modern GCs keep the performance penalty reasonably low. Still, you want to think about memory allocation, so that you don't allocate inefficiently. What GC delivers is first of all the guarantee of correctness, a pointer is either nil or points to a validly allocated memory. Also, it removes most needs of bookkeeping. That simplifies programs and especially allows to write clean APIs, where functions are fine to allocate reasonable amounts of memory. But wherever there are large parts of memory required, especially with a clear lifetime, I think a lot about memory allocation and even in a GC language I try to reuse memory wherever that makes sense. So for a game engine, there should not be much need for allocation during the run time of a "level" and thus GC pauses should only happen between levels.
- barrkel 7y ago
  Not thinking about ownership over references to heap allocations is very very much the big productivity win from GC. Memory safety is usually an additional bonus, but it's a distinct thing; it's possible to have (conservative or refcounting) GC without memory safety and vice versa.

brighteyes 7y ago

You're right, yes - a 1ms pause has a cost, even in a 16ms budget. I interpreted the author's point as "GC is now potentially viable for games", in the sense that older GCs might have pause times larger than 16ms, making them obviously inappropriate, whereas today 1ms pauses is something that can be budgeted for at least in some cases. As you said, malloc/free also have costs, and also GC has other benefits, like bump allocation, moving things to compact memory and improve cache locality, etc. 1ms pauses means GC is worth considering even for a game, in other words.

walkingolof 7y ago
"Games" is a term for are a large collection of very different types of software, do we mean AAA FPS games? Then I would probably not use a GC, if we mean most other games? Then it's just fine. Many GC events are way shorter than 1ms also...
- shrimp_emoji 7y ago
  Turn-based or pausable-real time 4X games where you spend most of your time paused and also get hit by a regular (~1000 ms) autosave freeze spring to mind.
- dfox 7y ago
  If you look around configuration REDEngine configuration files in PC version of Witcher 3, there is file with what for me looks like tuning parameters of Boehm-like conservative tracing GC, it is probably the only file in there that even includes comments about what the parameters mean and partially how they were determined. My overall impression from the various configuration files is that the whole world streaming/demand-paging mechanism is somehow interwoven with or even partially driven by the GC (so the GC is not meant for some small-ish scripting heap)
arcticbull 7y ago
> As you said, malloc/free also have costs, and also GC has other benefits, like bump allocation, moving things to compact memory and improve cache locality, etc. 1ms pauses means GC is worth considering even for a game, in other words. The general approach used by games is probably superior, which are arenas or zones. One approach games use is to allocate everything that's necessary for a given frame in a single large block, carve it off over time as needed, then drop the entire arena. All allocations then have memory locality, no fragmentation, basically zero cost to allocate (increment a pointer) and zero cost to deallocate. This kind of thing actually plays really nicely with Rust's lifetimes since you can couple frame lifetime with objects in that frame and get static validation. Arenas are already available in nightly [1]. [1] https://doc.rust-lang.org/nightly/nightly-rustc/arena/index.html https://doc.rust-lang.org/nightly/nightly-rustc/arena/index....

cmrdporcupine 7y ago

The other question is: can I at least time this GC so it doesn't overlap the refresh interval and skip a frame? Because 1ms may not be significant if it lands in the middle of the 16ms period, but if it lands at the end you've just produced jitter. So I'd think a GC designed for a gaming system would take this into account; incremental concurrent GC that can be timed to happen at the start of each each frame interval and be paused if it gets close to the end.

corysama 7y ago
You can manually trigger GC in Lua. I’ve used this to trigger light GC work every frame in some games to buy more consistent GC times (no spikes) at the cost of higher average times.
- brighteyes 7y ago
  Chrome does this too, it times GCs for when it has free time or is rendering in another thread, https://blog.chromium.org/2015/12/smarter-garbage-collection-for-smoother.html https://blog.chromium.org/2015/12/smarter-garbage-collection...
markdog12 7y ago
I remember the oddly named System.pauseForGCIfCollectionImminent() AS3 (Flash) function: https://help.adobe.com/en_US/FlashPlatform/reference/actionscript/3/flash/system/System.html#pauseForGCIfCollectionImminent() https://help.adobe.com/en_US/FlashPlatform/reference/actions...

skybrian 7y ago

Would it make a difference for Minecraft? I haven't played in a while but I remember fiddling with JVM settings to try to get smoother performance.

pjmlp 7y ago
I lost the link to the forum post, where it was described how bad Minecraft data structures were designed.
- blattimwind 7y ago
  Many years ago, when I used to play Minecraft, the common wisdom was simply to get as much memory as possible, so that the GC runs less frequently. With the debug overlay you could see how the game kept allocating at about a rate of 300-500 MB/s until the JVM GC was triggered, which created a noticeable lag in the game. I used to know a few MC server admins, and I was surprised just how massive the resource consumption of the game was. One server I usually played on ran on multiple machines. Specs were I believe around 128 GB memory, 32 cores, 16 or so dedicated to GC...
  - imtringued 7y ago
    Minecraft world simulation runs on a single core so you won't see much benefit from more cores. GC may be an exception though.
  - pjmlp 7y ago
    Something I remember from that forum post was that instead of using array based representation for the meshes, it uses classes for every kind of information (Vector, Points,...). Very nice school OOP, but not always the best way.

badsectoracula 7y ago

Considering how variable game frames are, even if the GC eats that 1ms it will be unnoticeable, assuming it doesn't happen too often. I mean, i play games like Skyrim that now and then give you a pause as you walk the world while it unloads cells (i refer to normal walking around, not entering/exiting somewhere - which, btw, would be a good place to force a GC too in a similar game that would be written in a GC'd language) which can even be 1 second long. 1ms is noise long lost in there.

blattimwind 7y ago
> Considering how variable game frames are, even if the GC eats that 1ms it will be unnoticeable, assuming it doesn't happen too often. For a competitive game you are looking at a refresh of 240 Hz (4 ms) but people generally target frame rates of at least 300-400 Hz (2.5 ms per frame!). There is simply no room to randomly throw in 1 ms pauses here and there without introducing micro-stuttering. Even for a casual game 1 ms of extra frame time variance is bad. If your Skyrim has one second freezes while roaming the world, your install is busted, because the game in proper working order doesn't do that.
- badsectoracula 7y ago
  > For a competitive game you are looking at a refresh of 240 Hz Yes, it depends on the game, what i wrote doesn't apply to every single game ever made out there, but it does apply to a ton of games. > Even for a casual game 1 ms of extra frame time variance is bad. If it happens every several minutes you will 100% not notice it at all. In pretty much every high end 3D game (especially an open world one) you'll get way more variance by turning the camera around and/or just walking than that. > If your Skyrim has one second freezes while roaming the world, your install is busted, because the game in proper working order doesn't do that. I'm 100% sure my Skyrim install is perfectly fine (it is a fresh one) and it does happen, just not frequently. If you haven't noticed it... well, as i already wrote, it isn't really noticeable. And - especially on Skyrim - you'll also get way more variance than 1ms just walking around.
  - 7y ago

pron 7y ago

Games have a very regular object lifetime, allowing them to manage much of their allocations in arenas, which can certainly beat even modern GCs in performance (although maybe not for good). But concurrent server business applications don't have such regular object lifetimes, and it is getting quite hard to beat the performance of modern GCs with manual memory management in such applications. These days, footprint is the major and pretty much only significant cost to GCs in a great many applications.

hinkley 7y ago

My second job was displaying telemetry waveforms in real time, in Java... 1.3 if memory serves. MVP goal was 4 waveforms, I managed to get 16. What we did to accomplish this was first aggressively optimize the data ingestion, then spend all that surplus on maintaining a ring buffer of pixels. This meant we were always a little behind live (.5s or 1s, I can't recall), but it made everything smooth as silk. To maintain homeostasis, I'd trigger GC every time we made it through a paint cycle with a full buffer. There are ways to do it, but it could change the structure of your app. I say, "in our case the main paint loop was stupid-simple so it wasn't that big of a deal to tack it on," but the idea was always sort of in the back of my mind so it definitely informed the design. What I didn't have to deal with is random inputs from the user changing my display strategy.