6 ms·
The p50 speed up from delaying memory management comes with a trade off, namely you get Garbage Collection pauses, bad p99, and spend your effort tuning the Gar
by nelsondev 4y ago
The p50 speed up from delaying memory management comes with a trade off, namely you get Garbage Collection pauses, bad p99, and spend your effort tuning the Garbage Collector instead of your code.
- xxpor 4y agoAn uncommon but not unheard method around that (in services that have diurnal patterns) is to throw memory at it and collect once a day during the down time. You can take the server out of service, restart it, and put it back in service.
- ska 4y agoI read this was semi-standard memory management in the early lisp machine days, because that gc was so slow.
- bitwize 4y agoEarly Lisp (even pre-lispm) had no GC at all -- it just allocated until it ran out. Memory reclamation was done by saving the heap to tape and then reading it back -- only live objects were saved. So it was kind of GC with extra steps; the first GC algorithms removed the steps.
- User23 4y agoTo this day the Common Lisp standard doesn’t require GC.
- lispm 4y agoNah, Lisp had GC in 1960 already. McCarthy invented the mark & sweep GC. Lisp I Programmers Manual, 1960 https://www.softwarepreservation.org/projects/LISP/book/LISP%20I%20Programmers%20Manual.pdf https://www.softwarepreservation.org/projects/LISP/book/LISP... Chapter 6.3, Page 95, The Free-Storage List and the Garbage Collector
- plandis 4y agoThis is kind of hard to do in practice unless you’re really watching where you’re allocating memory. It’s very easy in Java to end up accidentally throwing a bunch of allocated memory on the heap without realizing it. Fwiw, the JVM now has a noop garbage collector so this is easy enough to benchmark.
- vbezhenar 4y agoCan you provide an examples of accidentally allocating memory? The only example I'm aware of is autoboxing which is huge misfeature IMO.
- kaba0 4y agoAny method call could allocate objects in their call trees, and it may not even behave consistently with new versions. But it is easy to monitor with Java’s killer observability.
- lenocinor 4y agoConcatenating lots of Strings instead of using a StringBuffer will do the trick. Sure, people who are familiar with Java know not to do that. But I’ve seen plenty of production code that concatenates.
- kaba0 4y agoDoes it really? With Java’s G1GC the only knob you may end up having to tune is the max target pause time - which pretty much chooses between better p50 and worse p99 va worse p50 and better p90 - aka throughput vs latency, that are almost inherently opposite ends of the same spectrum. It is not GC-specific that any improvement will fail to increase one of those. So the actual tradeoff is more whether you want better throughput or tail-latency. To improve the latter, you have a singular command line option of using ZGC as well.
- hedora 4y agoYou mean the integer parameter "-XX:MaxGCPauseMillis"? The name says it all. You can do over a dozen sync disk writes in a millisecond these days.
- kaba0 4y agoHave you seen the output of JVM flight recorder or some other tool used to observe memory allocation/reclamation patterns? The JVM is very lazy when it comes to GC - its algorithms are crazy good and can catch up with really high allocation rates as well. So the maximum pause time’s unit doesn’t really say much, and when it comes to dynamic memory allocation, the JVM is the best choice according to the biggest web services operating on terabyte sized heaps.
- kasperni 4y agoWith modern concurrent GCs such as ZGC [1] this is really not an issue for most people anymore. [1] https://inside.java/2022/05/30/sip053/#:~:text=ZGC%20was%20designed%20as%20a,a%20maximum%20of%2016%20TBs https://inside.java/2022/05/30/sip053/#:~:text=ZGC%20was%20d....
- philipkglass 4y agoWith Java 17 I found that using ZGC eliminated [1] the GC pauses that yield bad P99 latencies without any additional tuning. However, it has lower throughput than G1GC so your P50 advantages will diminish. [1] Reduced them to always well below 1 millisecond.
- cogman10 4y agoWhich is always the tradeoff with GC. You get to pick latency or throughput and you trade one for the other.
- kaba0 4y agoIn my experience, this is not related closely to GC and are apparent in many systems. A great book on optimizations explicitly mentions that “improving one aspect of performance may degrade another”.
- plandis 4y agoLow pause collectors exist explicitly for this purpose. ZGC and Shenandoah both trade throughput (more concurrent collections) for extremely low pause times: https://malloc.se/blog/zgc-jdk16 https://malloc.se/blog/zgc-jdk16
- nimrody 4y agoDoes that mean they also trade _memory_ for low pause times? Because, in our application, the memory required for avoiding significant GC pauses is at least 4x the amount of memory we actually need at any given time. This can be significant too.
- mabbo 4y agoThat sounds a lot like the complaints about Java from a decade ago. Have you used it lately? Java's GCs are incredible and you have a menu of algorithmic options that let you avoid whatever problem you're worried about.
- dboreham 4y agoThe JVM GC is fundamentally the same as 10 years ago.
- titzer 4y agoShenandoah did not exist 10 years ago, and G1 was not deployed.
- bitcharmer 4y agoSaying this just shows you don't know much about modern JVM.
- hackerlight 4y agoDo you use a proprietary JVM at your place?
- bitcharmer 4y agoNope. Only OpenJDK and Azul Prime.
- cogman10 4y ago10 years ago the algorithms you could pick were serial, parallel, and CMS. Today we've removed CMS, added G1GC, and added ZGC. G1GC is similar to the parallel in the way it operates but is divided up enough to control for latency. ZGC is fundamentally different from the parallel collector or G1GC. Suggesting it is the same is to suggest you know nothing of the changes that have happened.
- jcelerier 4y ago
- titzer 4y ago> tuning the Garbage Collector instead of your code. This is a feature, not a bug. Which is riskier?
- cogman10 4y agoNow-a-days, if you are tuning your GC you are almost certainly doing the wrong thing. While the JVM offers a plethora of levers to pull in case you are hyper concerned about different things, the heuristics are VERY good. Mucking with the fine grained details can disable heuristics and ultimately give you worse results than if you just left stuff alone. The levers to pull are algorithm, max memory, and max pause time. All other levers should be left alone unless you've got GC logs to back up what you think needs changing. (And even then... Do you really?) Typically, the better route is flight recorder and eliminating wasted allocations.
- xxs 4y ago> Garbage Collection pauses True concurrent GC have been available since late 2000s. Azul had a read-barrier GC as well - effectively a pauseless GC (or pauses under 1ms)