3 ms·
Good observation. This is a fundamental principle of garbage collection. You can either have low latency or high throughput. You can't get both. Why is that
by cogman10 6y ago
Good observation.
This is a fundamental principle of garbage collection. You can either have low latency or high throughput. You can't get both.
Why is that?
All optimizations that improve latency come at a cost. Generally, more book keeping, more checks, more frequent garbage collections. ZGC is one of those algorithms. It adds a new check every time you access memory to see if it needs to be relocated. That increases the size of objects but also the general runtime of the application.
A similar thing happens with reference counting (which is on the extreme end of the latency/throughput tradeoff). Every time you give a shared pointer or release a shared pointer a check is performed to see if a final release needs to happen.
On the flip side, a naive mark and sweep algorithm is trivially parallelizable. The number of times you check if memory is still in use is bound by when a collection happens. In an ideal state you increase heap size until you get the desired throughput.
We get "violations" of some of these principles if we can take shortcuts or have assumptions about how memory is used and allocated. For example, the assumption that "most allocations are short lived" or the generational hypotheses leads to shorter pause times even when optimizing for throughput without a lot of extra cost. It's only costly when you've got an application that doesn't fit into that hypotheses (which is rare).
Haskell has a somewhat unique garbage collector based on the fact that all data is immutable. They can take shortcuts because older references can't refer to newer references.
- The_rationalist 6y agoHaskell has a somewhat unique garbage collector based on the fact that all data is immutable. They can take shortcuts because older references can't refer to newer references. I wonder, can this be achieved for immutable datastructures in the JVM e.g records, lists ?
- deleted 6y ago[deleted]
- pron 6y ago> Haskell has a somewhat unique garbage collector based on the fact that all data is immutable. They can take shortcuts because older references can't refer to newer references. When you don't mutate in OpenJDK you get essentially the same. Much of the cost of a modern GC (OpenJDK's G1, and soon probably ZGC, too) is write barriers, that need to inform the GC about reference mutations. If you don't mutate, you don't pay that cost. This is partly why applications that go to the extreme in the effort not to allocate and end up mutating more, might actually do worse than if they'd allocated more with OpenJDK's newer GCs. In fact, OpenJDK's GCs rely heavily on the assumption that old objects can't reference newer ones unless explicitly mutated, and so require those barriers only in old regions.
- chmod775 6y ago> This is partly why applications that go to the extreme in the effort not to allocate and end up mutating more. If you go to the extreme and don't allocate, you can turn the GC off.
- pron 6y agoYes (well, sort of; you could use the Epsilon GC, which is essentially a no-op GC), but I once read of a library that allocated all of its objects at initialisation and claimed to be "GC-neutral", i.e. not add any memory management burden regardless of the GC chosen by the application using it. They were very surprised when they tried running their benchmarks with G1. OpenJDK's modern GCs are optimised for "reasonable behaviour." Allocate too much and you suffer; allocate too little (by which I mean reuse objects and mutate them a lot) and you also suffer. But as long as your behaviour is in the broad more-or-less normal range, you get some really good performance, which is only getting better with each release.
- whateveracct 6y ago> Haskell has a somewhat unique garbage collector based on the fact that all data is immutable. They can take shortcuts because older references can't refer to newer references. I don't think this is true? Because laziness is really heavily mutable under the hood. Not to mention that it has mutable references. But maybe there are some tricks in the GC I'm not aware of.
- cogman10 6y agoIf you're interested in a fun read, they've published a paper on how they do garbage collection. http://simonmar.github.io/bib/papers/parallel-gc.pdf http://simonmar.github.io/bib/papers/parallel-gc.pdf
- moonchild 6y ago> This is a fundamental principle of garbage collection. You can either have low latency or high throughput. You can't get both. > All optimizations that improve latency come at a cost Huh? That's a very strange, absolutist statement. There are many cases where you have the opportunity to trade off one of latency and throughput for the other, yes. But there are also many cases where an optimization can improve both.
- cogman10 6y agoOptimization is probably the wrong word, fundamental algorithm might be a better one. For example, I certainly could see some SIMD optimization improving latency and throughput, that's not what I'm talking about. I'm talking about the overarching GC algorithm. AFAIK, there's no algorithm that's both good at throughput and latency without making assumptions around how memory is used.