4 ms·

"Optimized" as in stops the user thread from doing useful work, when a lot of allocation happens? At least a couple of years ago Go had a very simplistic GC, b

by gf000 2y ago

"Optimized" as in stops the user thread from doing useful work, when a lot of allocation happens? At least a couple of years ago Go had a very simplistic GC, but even today it is absolutely nowhere close to how good Java's GCs are (there is ZGC, for example, whose pause times are completely decoupled from heap size, so it can actually keep sub-millisecond pause times - the OS causes bigger pauses than that). At most Go puts slightly less work on its GC due to having value types. Sure, it's a "microbenchmark", but it might be eye-opening how big the difference is: https://benchmarksgame-team.pages.debian.net/benchmarksgame/performance/binarytrees.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

neonsunset 2y ago
Binary-trees is very interesting. Showcases how Java can easily compete with native code that uses hand-managed arena allocators. While investigating why it is so much better than .NET* in this case I reached a conclusion that one of the major contributing factors is that it uses TLAB which "inlines" the GC allocation code completely into the callers, making the allocations indeed just thread-local pointer bumps. .NET has something similar (allocation context) but you do have to go through a call. I assume TLAB allocations and TLAB refills are just this much better regardless. (* in all the number crunching ones we have every other GC language, including Java and Go, confidently beat <3 )
- gf000 2y ago
  Yeah, it's probably a bit unfair given that GC research itself is just Java. Also, there might be some philosophical difference at play here, Java (the JVM) tends to expose only a very limited API (no structs, pointers, etc), but this allows more flexibility on the runtime part, while .NET let's the developer touch/control everything, but that might means less room for doing some advanced stuff on the runtime side. The age-old generic-specific balance. (Also, I like this playful competition between languages, much better than pointless flamewars :D)
mknyszek 2y ago
Looking at only the CPU numbers from this benchmark is misleading. This site requires the use of default configurations for each language runtime, and JVMs tend to have a much larger default heap than the Go runtime. Tracing GCs tend to have a CPU/memory tradeoff built into them [1]. Compare the memory footprint of the best Go and best Java programs in terms of wall time [2] (the site doesn't make it easy, you have to go back and forth between the two links) and the difference is enormous (these Go programs are running with much smaller total heap sizes, so much less runway for the GC). If you use GOGC and GOMEMLIMIT to even the playing field (and note, use a Go program that isn't using sync.Pool) the difference in wall time is far less stark (though it's still there, maybe 5-15%; don't quote me, it's been a long time since I measured this and I don't remember exactly). (The difference in total CPU time is bigger.) And finally, keep in mind this benchmark is hammering as hard as it can on the GC. How it impacts real applications depends on how much the application relies on the heap. [1] https://go.dev/doc/gc-guide#Understanding_costs https://go.dev/doc/gc-guide#Understanding_costs [2] https://benchmarksgame-team.pages.debian.net/benchmarksgame/performance/binarytrees-mem.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
- gf000 2y ago
  I can see a ~3x difference in memory usage in case of JIT-compiled Java vs Go, but AOT-compiled Java comes in at less memory usage while still being faster than Go. I believe Java AOT uses the Serial GC, while the "normal" version defaults to G1, so there is that (the GC code is actually reused between Graal and OpenJDK, so we could remove this "variable") Don't forget that the JVM has to allocate memory for all its subsystems as well, like the JIT compiler, so that 3x memory is not entirely heap usable by the program. And I deliberately linked this benchmark, as the topic at hand is the GC itself.
  - mknyszek 2y ago
    Fascinating. I could see the Serial GC, if it's generational, just totally crush this particular benchmark. I wonder what the heap size heuristic is for the Serial GC. > Don't forget that the JVM has to allocate memory for all its subsystems as well, like the JIT compiler, so that 3x memory is not entirely heap usable by the program. That's fair. I recall doing my due diligence here and confirming it is actually using mostly heap memory, but again it's been a while and I could be wrong. (Also if the actual heap size is only ~100s of MiB and the rest of the subsystems need north of a GiB, that's much more than I would have anticipated.) > And I deliberately linked this benchmark, as the topic at hand is the GC itself. Sure. Not trying to suggest the benchmark doesn't have any utility, just that even for just GC performance, it doesn't paint a complete picture, especially if you're only looking at wall time.

by gf000 2y ago

neonsunset 2y ago

Binary-trees is very interesting. Showcases how Java can easily compete with native code that uses hand-managed arena allocators. While investigating why it is so much better than .NET* in this case I reached a conclusion that one of the major contributing factors is that it uses TLAB which "inlines" the GC allocation code completely into the callers, making the allocations indeed just thread-local pointer bumps. .NET has something similar (allocation context) but you do have to go through a call. I assume TLAB allocations and TLAB refills are just this much better regardless. (* in all the number crunching ones we have every other GC language, including Java and Go, confidently beat <3 )

gf000 2y ago
Yeah, it's probably a bit unfair given that GC research itself is just Java. Also, there might be some philosophical difference at play here, Java (the JVM) tends to expose only a very limited API (no structs, pointers, etc), but this allows more flexibility on the runtime part, while .NET let's the developer touch/control everything, but that might means less room for doing some advanced stuff on the runtime side. The age-old generic-specific balance. (Also, I like this playful competition between languages, much better than pointless flamewars :D)

mknyszek 2y ago

Looking at only the CPU numbers from this benchmark is misleading. This site requires the use of default configurations for each language runtime, and JVMs tend to have a much larger default heap than the Go runtime. Tracing GCs tend to have a CPU/memory tradeoff built into them [1]. Compare the memory footprint of the best Go and best Java programs in terms of wall time [2] (the site doesn't make it easy, you have to go back and forth between the two links) and the difference is enormous (these Go programs are running with much smaller total heap sizes, so much less runway for the GC). If you use GOGC and GOMEMLIMIT to even the playing field (and note, use a Go program that isn't using sync.Pool) the difference in wall time is far less stark (though it's still there, maybe 5-15%; don't quote me, it's been a long time since I measured this and I don't remember exactly). (The difference in total CPU time is bigger.) And finally, keep in mind this benchmark is hammering as hard as it can on the GC. How it impacts real applications depends on how much the application relies on the heap. [1] https://go.dev/doc/gc-guide#Understanding_costs https://go.dev/doc/gc-guide#Understanding_costs [2] https://benchmarksgame-team.pages.debian.net/benchmarksgame/performance/binarytrees-mem.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

gf000 2y ago
I can see a ~3x difference in memory usage in case of JIT-compiled Java vs Go, but AOT-compiled Java comes in at less memory usage while still being faster than Go. I believe Java AOT uses the Serial GC, while the "normal" version defaults to G1, so there is that (the GC code is actually reused between Graal and OpenJDK, so we could remove this "variable") Don't forget that the JVM has to allocate memory for all its subsystems as well, like the JIT compiler, so that 3x memory is not entirely heap usable by the program. And I deliberately linked this benchmark, as the topic at hand is the GC itself.
- mknyszek 2y ago
  Fascinating. I could see the Serial GC, if it's generational, just totally crush this particular benchmark. I wonder what the heap size heuristic is for the Serial GC. > Don't forget that the JVM has to allocate memory for all its subsystems as well, like the JIT compiler, so that 3x memory is not entirely heap usable by the program. That's fair. I recall doing my due diligence here and confirming it is actually using mostly heap memory, but again it's been a while and I could be wrong. (Also if the actual heap size is only ~100s of MiB and the rest of the subsystems need north of a GiB, that's much more than I would have anticipated.) > And I deliberately linked this benchmark, as the topic at hand is the GC itself. Sure. Not trying to suggest the benchmark doesn't have any utility, just that even for just GC performance, it doesn't paint a complete picture, especially if you're only looking at wall time.