8 ms·

1 ms pause times are pretty good. That's finally getting close to the point where it may no longer be the biggest factor preventing adoption in applications lik

1 ms pause times are pretty good. That's finally getting close to the point where it may no longer be the biggest factor preventing adoption in applications like core game engine code. Although at 144 Hz it's still 14% of your frame time, so it's hardly negligible. Even if the GC is running on an otherwise idle core there are still other costs like power consumption and memory bandwidth. So you still want to minimize allocation to keep the GC workload down. For too long GC people were touting 10 ms pause times as "low" and not bothering to go further, but truly low pause times are possible. I'd love to see a new systems language that starts by designing for extremely low-pause GC, not manual allocation or a borrow checker. I think it would be possible to make something that you could use for real time work without having to compromise on memory safety and without having to pay the complexity tax Rust takes on for the borrow checker.

The_rationalist 6y ago
Although at 144 Hz it's still 14% of your frame time well if we believe their numbers, the worst case is 0.5ms so 6% of frame time for 144hz. Assuming their stated average pause time of 0.05ms then average pauses (and the GC isn't constantly pausing) take 0.6% of frametime, which is negligible. Though your concerns on throughput and resource usages stands. Well newer programming languages could leverage ZGC (and improve upon it) by targeting graalVM + it enable cross-language interop.
- modeless 6y ago
  In my experience GC developers wildly underestimate their worst case, so I don't really believe that 0.5ms number. But more importantly, you should not use average pause time at all. At 144 Hz the 99th percentile frame time occurs more than once per second. If you want to avoid dropping frames you need to design for the worst case.
  - kaba0 6y ago
    ZGC used to target 10ms as worst-case latency, and they target 3-4ms now I believe.
    - jacques_chester 6y ago
      The article above says that the target is 1ms.
      - kaba0 6y ago
        Thanks for the correction, I remembered the 3-4 ms from the inside java podcast on the ZGC.
        jacques_chester 6y ago
        No worries.
  - doikor 6y ago
    There is a worst case that is much worse then the now mentioned 1ms. Namely really big change in allocation rate in which case the gc cycle does not finish before running out of memory (ZGC uses allocation rate to determine when to start next gc cycle). In that case ZGC stops allocations ("Allocation Stall" in gc logs). But this is basically a failure mode and should not happen during normal operations at all. Though you can configure for this in a couple ways if you run into this issue 1. By telling it to treat some amount of heap as the max that is not actual max when it comes to it calculating when it should start next gc cycle. (-XX:SoftMaxHeapSize) https://malloc.se/blog/zgc-softmaxheapsize https://malloc.se/blog/zgc-softmaxheapsize 2. Increasing the amount of concurrent gc threads so they will finish their work faster (-XX:ConcGCThreads) 3. Just run really large heap so the "run a gc cycle even if we don't need to based on allocation rate" gc cycle keeps the heap in check. Though after JDK 15 we have not had to mess with any of these. Prior to that we had to adjust soft max heap size a bit. With JDK 16 it should be even better I guess (should be upgrading sometime next week)
moonchild 6y ago
> at 144 Hz it's still 14% of your frame time, so it's hardly negligible A single skipped frame is not a big deal and will probably not be noticed. It will probably happen anyway due to scheduling quirks, resource contention with other processes, existing variation in frametime... True realtime work requires no dynamic allocations whatsoever (which, notably, is not covariant with gc!), so I think ‘low’ pause times are an acceptable compromise. Where performance is a concern, you need to manually manage a lot of factors, among them GC/dynamic memory use. There's no runtime that can obviate that. Granted, 1ms pause times are probably still not low enough for realtime audio, and there may be room for some carelessness there (audio being soft realtime, not hard realtime). But I think just being careful to avoid dynamic allocation on the audio thread is probably a worthwhile tradeoff.
- modeless 6y ago
  > A single skipped frame is not a big deal and will probably not be noticed. Attitudes like this are why my phone sucks to use and I get nauseous in VR and GC devs spent so long in denial saying 10 ms pause times should be good enough. Yes, single dropped frames matter. If you don't think so then I don't want to use your software.
  - kaba0 6y ago
    A single skipped frame usually means that we are talking about soft real time. And there it is absolutely acceptable, not in the average case, but eg. on a heavily used computer a slight drop in audio is “appropriate”, it’s not an anti-missile device. It won’t make the normal case jittery, nauseus or anything like that. Also, in regards to your GC devs comment, I would say that attitudes like this is the problem.. The great majority of programs can do with much more than 10 ms pause times.
    - akx 6y ago
      A slight drop in audio would be perfectly unacceptable for eg computers running concerts.
Thaxll 6y ago
GC will most likely never be used in demanding games. You want total control over memory. 1ms sounds ok but still you don't know when and for how long the GC is going to kicks in.
- BenoitP 6y ago
  > for how long the GC is going to kicks in. 1ms (max, average at 50us) And for the 'when' I'll add that the very concept of having a concurrent GC means you don't need to do a (potentially pausing) malloc right in the middle of what you're trying to do.
  - monocasa 6y ago
    The kind of people that care about GC pause times have their own allocators that are as cheap as jvm allocations and cheaper deallocation. They aren't poopooing GC's and then just calling regular malloc and free.
  - Thaxll 6y ago
    Engines rely on smart allocator and memory pool, they usually allocate everything beforehand. You're not running malloc between two frames. Imagine a game like Battlefield if you were to allocate memory for each fired bullet.
- CJefferson 6y ago
  The biggest engine in gaming, Unity, uses c#, which is GCed.
  - Thaxll 6y ago
    It's def not the biggest, it's almost not used in "AAA" games also.
    - liamkf 6y ago
      Unreal also has a GC to deal with. I've spent more time on AAA games than I'd like to admit trying to hide/mitigate/optimize the hitch.
higerordermap 6y ago
At this point, I wonder whether it's possible to architect an efficient GC that predictably runs only when alloc() is called and thus helps to avoid pauses in main loop. Of course it works best with a language having value types.
- dan-robertson 6y ago
  This is the standard way traditional incremental GC works. Typically allocation is done with a protocol like 1. Try to bump a pointer to allocate the memory 2. If there wasn’t enough memory, call a special function (which runs the GC) and try again This special function may do a big gc pause if necessary or it may just do a small amount of gc work and grant the program some more memory (this way most gc work is shared more evenly with program time). In modern VMs like Java’s, concurrent gc (where gc and mutator run in separate threads possibly needing a pause for some steps) is more popular.
dan-robertson 6y ago
I think games are just a bad example. Many games have both simple ownership semantics (making manual memory management easier) and simple object lifetimes. In a game an object is likely to live for exactly one frame, exactly n frames where n is like 2 or 3 for some kind of deferred rendering, or an entire level. Some games mightn’t have levels as such but objects still tend to be grouped together in when they load/unload. I think it is important to note this because games that need to care about performance are not using malloc. They will use some kind of arena allocation because allocating and freeing memory with malloc can be expensive. So what games want here is control over how allocation happens rather than when free is called—that is, it wouldn’t be sufficient to have a language where you called free but couldn’t use a custom allocator. But there are standard ways to deal with these sorts of allocation patterns in a language like Java. For buffers (of eg vertices) these may just be reused. For smaller objects, pool allocation may be used which sucks but doesn’t suck much more than manual memory management. I think it is silly to focus on games (or, for that matter, short-running command-line tools that needn’t call free at all) because the allocation pattern is so unusual and weird. I can’t think of great examples of systems that care about latency and don’t have game-like allocation patterns but maybe a complex gui application like a web browser or PowerPoint would be a good example.
- kllrnohj 6y ago
  Games also care heavily about data layout, which you can't really do in a GC'd language very well. Well, game engines care heavily about that, anyway. Pooling objects doesn't get you there.
  - thu2111 6y ago
    There's no tradeoff between using a GC and controlling data layout. CLR already allows value types and JVM is getting support for advanced value types and specialised generics (templates) over them.

The_rationalist 6y ago

Although at 144 Hz it's still 14% of your frame time well if we believe their numbers, the worst case is 0.5ms so 6% of frame time for 144hz. Assuming their stated average pause time of 0.05ms then average pauses (and the GC isn't constantly pausing) take 0.6% of frametime, which is negligible. Though your concerns on throughput and resource usages stands. Well newer programming languages could leverage ZGC (and improve upon it) by targeting graalVM + it enable cross-language interop.

modeless 6y ago
In my experience GC developers wildly underestimate their worst case, so I don't really believe that 0.5ms number. But more importantly, you should not use average pause time at all. At 144 Hz the 99th percentile frame time occurs more than once per second. If you want to avoid dropping frames you need to design for the worst case.
- kaba0 6y ago
  ZGC used to target 10ms as worst-case latency, and they target 3-4ms now I believe.
  - jacques_chester 6y ago
    The article above says that the target is 1ms.
    - kaba0 6y ago
      Thanks for the correction, I remembered the 3-4 ms from the inside java podcast on the ZGC.
      - jacques_chester 6y ago
        No worries.
- doikor 6y ago
  There is a worst case that is much worse then the now mentioned 1ms. Namely really big change in allocation rate in which case the gc cycle does not finish before running out of memory (ZGC uses allocation rate to determine when to start next gc cycle). In that case ZGC stops allocations ("Allocation Stall" in gc logs). But this is basically a failure mode and should not happen during normal operations at all. Though you can configure for this in a couple ways if you run into this issue 1. By telling it to treat some amount of heap as the max that is not actual max when it comes to it calculating when it should start next gc cycle. (-XX:SoftMaxHeapSize) https://malloc.se/blog/zgc-softmaxheapsize https://malloc.se/blog/zgc-softmaxheapsize 2. Increasing the amount of concurrent gc threads so they will finish their work faster (-XX:ConcGCThreads) 3. Just run really large heap so the "run a gc cycle even if we don't need to based on allocation rate" gc cycle keeps the heap in check. Though after JDK 15 we have not had to mess with any of these. Prior to that we had to adjust soft max heap size a bit. With JDK 16 it should be even better I guess (should be upgrading sometime next week)

moonchild 6y ago

> at 144 Hz it's still 14% of your frame time, so it's hardly negligible A single skipped frame is not a big deal and will probably not be noticed. It will probably happen anyway due to scheduling quirks, resource contention with other processes, existing variation in frametime... True realtime work requires no dynamic allocations whatsoever (which, notably, is not covariant with gc!), so I think ‘low’ pause times are an acceptable compromise. Where performance is a concern, you need to manually manage a lot of factors, among them GC/dynamic memory use. There's no runtime that can obviate that. Granted, 1ms pause times are probably still not low enough for realtime audio, and there may be room for some carelessness there (audio being soft realtime, not hard realtime). But I think just being careful to avoid dynamic allocation on the audio thread is probably a worthwhile tradeoff.

modeless 6y ago
> A single skipped frame is not a big deal and will probably not be noticed. Attitudes like this are why my phone sucks to use and I get nauseous in VR and GC devs spent so long in denial saying 10 ms pause times should be good enough. Yes, single dropped frames matter. If you don't think so then I don't want to use your software.
- kaba0 6y ago
  A single skipped frame usually means that we are talking about soft real time. And there it is absolutely acceptable, not in the average case, but eg. on a heavily used computer a slight drop in audio is “appropriate”, it’s not an anti-missile device. It won’t make the normal case jittery, nauseus or anything like that. Also, in regards to your GC devs comment, I would say that attitudes like this is the problem.. The great majority of programs can do with much more than 10 ms pause times.
  - akx 6y ago
    A slight drop in audio would be perfectly unacceptable for eg computers running concerts.

Thaxll 6y ago

GC will most likely never be used in demanding games. You want total control over memory. 1ms sounds ok but still you don't know when and for how long the GC is going to kicks in.

BenoitP 6y ago
> for how long the GC is going to kicks in. 1ms (max, average at 50us) And for the 'when' I'll add that the very concept of having a concurrent GC means you don't need to do a (potentially pausing) malloc right in the middle of what you're trying to do.
- monocasa 6y ago
  The kind of people that care about GC pause times have their own allocators that are as cheap as jvm allocations and cheaper deallocation. They aren't poopooing GC's and then just calling regular malloc and free.
- Thaxll 6y ago
  Engines rely on smart allocator and memory pool, they usually allocate everything beforehand. You're not running malloc between two frames. Imagine a game like Battlefield if you were to allocate memory for each fired bullet.
CJefferson 6y ago
The biggest engine in gaming, Unity, uses c#, which is GCed.
- Thaxll 6y ago
  It's def not the biggest, it's almost not used in "AAA" games also.
  - liamkf 6y ago
    Unreal also has a GC to deal with. I've spent more time on AAA games than I'd like to admit trying to hide/mitigate/optimize the hitch.

higerordermap 6y ago

At this point, I wonder whether it's possible to architect an efficient GC that predictably runs only when alloc() is called and thus helps to avoid pauses in main loop. Of course it works best with a language having value types.

dan-robertson 6y ago
This is the standard way traditional incremental GC works. Typically allocation is done with a protocol like 1. Try to bump a pointer to allocate the memory 2. If there wasn’t enough memory, call a special function (which runs the GC) and try again This special function may do a big gc pause if necessary or it may just do a small amount of gc work and grant the program some more memory (this way most gc work is shared more evenly with program time). In modern VMs like Java’s, concurrent gc (where gc and mutator run in separate threads possibly needing a pause for some steps) is more popular.

dan-robertson 6y ago

I think games are just a bad example. Many games have both simple ownership semantics (making manual memory management easier) and simple object lifetimes. In a game an object is likely to live for exactly one frame, exactly n frames where n is like 2 or 3 for some kind of deferred rendering, or an entire level. Some games mightn’t have levels as such but objects still tend to be grouped together in when they load/unload. I think it is important to note this because games that need to care about performance are not using malloc. They will use some kind of arena allocation because allocating and freeing memory with malloc can be expensive. So what games want here is control over how allocation happens rather than when free is called—that is, it wouldn’t be sufficient to have a language where you called free but couldn’t use a custom allocator. But there are standard ways to deal with these sorts of allocation patterns in a language like Java. For buffers (of eg vertices) these may just be reused. For smaller objects, pool allocation may be used which sucks but doesn’t suck much more than manual memory management. I think it is silly to focus on games (or, for that matter, short-running command-line tools that needn’t call free at all) because the allocation pattern is so unusual and weird. I can’t think of great examples of systems that care about latency and don’t have game-like allocation patterns but maybe a complex gui application like a web browser or PowerPoint would be a good example.

kllrnohj 6y ago
Games also care heavily about data layout, which you can't really do in a GC'd language very well. Well, game engines care heavily about that, anyway. Pooling objects doesn't get you there.
- thu2111 6y ago
  There's no tradeoff between using a GC and controlling data layout. CLR already allows value types and JVM is getting support for advanced value types and specialised generics (templates) over them.