5 ms·
> A key design insight for LXR is that regular brief STW pauses provide a highly-efficient, low-latency approach to garbage collection. I have been playing aro
by bob1029 5y ago
> A key design insight for LXR is that regular brief STW pauses provide a highly-efficient, low-latency approach to garbage collection.
I have been playing around with low-latency GC strategies in .NET6, and have discovered a similar pattern - If you intentionally run GC as frequently as makes sense (i.e. instead of trying to avoid it or sweep it under some other rug), you can sometimes reach a solution that is far superior to any other.
My example being a toy software rendering engine that is utilized by several concurrent clients. I asked myself "What would happen if I just ran GC right before every single frame"? Turns out, for reasonable scene sizes, the maximum amount of time the GC pause lasted was <5ms, producing a perfect result for my use case. Without intentionally calling GC, I would sometimes see pauses upwards of 3-4 seconds, which clearly has a severe impact on user experience for this type of application.
The way it works in my head - I am happy to pay a small 1~5ms cost for every frame (or batch of frames as the case may be) rather than allowing the .NET runtime to decide when would be most convenient and incur a much longer collection.
"Clean as you go" might be the best way to achieve low latency in any reality. Deferring and hiding is how things build up. Something is gonna have to eventually deal with all that trash unless you are shipping cruise missiles.
- zozbot234 5y agoOf course, the limiting case of "clean as you go" is either fully manual memory management, or at least deterministic deallocation (as seen in reference counting systems). With some exceptions for small bursts of especially low-latency, high-throughput work, that can be modeled in a "manual" system by arenas, or higher-scoped references if using reference counting.
- eru 5y agoAlas, reference counting and C-style manual memory management can still require doing arbitrary amounts of work from time to time.
- deleted 5y ago[deleted]
- kaba0 5y agoI don’t believe it is as easy as that. What happens when your program have accumulated plenty of long-living objects? Your case probably works due to having only objects with short lifetimes - but your GC’s pause time will generally grow with the heap size. Not even generational GCs solve that in the general case. Java’s low-latency GC, ZGC moves almost the whole work to parallel threads by accepting the trade-off of slightly slower barriers. But in reward it can decouple the pause time’s length from the heap size which is the key here.
- adgjlsfhk1 5y agoThe whole point of generational GC is that that your pause times don't grow with the old objects. I think concurrent sweep makes a lot of sense (it's basically free), but concurrent mark has major performance compromises. It's not just the longer barrier, it also adds an extra indirection to all your memory accesses. Full concurrent may be right for some use cases (where you *need* very low pause times), but it comes at a considerable cost.
- kaba0 5y agoI don’t think that generational GC can totally decouple pause times from heap size in the worst case, which is what is important for latency-oriented workloads.
- adgjlsfhk1 5y agoFor stuff like games, it probably does it well enough. If you drop a frame every minute, that's not a major problem.
- jhgb 5y ago> My example being a toy software rendering engine that is utilized by several concurrent clients. I asked myself "What would happen if I just ran GC right before every single frame"? Turns out, for reasonable scene sizes, the maximum amount of time the GC pause lasted was <5ms, producing a perfect result for my use case. Without intentionally calling GC, I would sometimes see pauses upwards of 3-4 seconds, which clearly has a severe impact on user experience for this type of application. Oberon, if I'm not mistaken, had stupidly simple GC that simply run between every two user interactions. That way you wouldn't notice pauses simply because your fingers move way slower than bytes in memory.