3 ms·

That's a common wisdom, but there is little if any evidence to support it. True GC costs are simply more hidden and harder to measure. A malloc/free pair might

That's a common wisdom, but there is little if any evidence to support it. True GC costs are simply more hidden and harder to measure. A malloc/free pair might be slightly slower than Java's new (not by much really - go measure), but invoking `new` is not the only cost of GC. https://arxiv.org/abs/2112.07880 https://arxiv.org/abs/2112.07880 Also modern low pause GCs have way higher CPU overhead than the old STW ones. Try to set a low pause GC pause target to 0.1 ms or set the heap limit to 110% of live memory and see what happens to CPU consumption. Last time I tried, the app didn't even start - got OOM nearly instantly. There is a reason the study above measures at 3x oversized heap. The cost of GC is lower than malloc/free only if you give it a lot more memory than you actually need. Which in many cases is a good tradeoff to make, but it is good to know such tradeoff between memory and CPU exists. And finally there is one more thing that's not particularly a trait of GC, but rather a limitation of current Java - it is much easier to write a C++/Rust program with low number of heap allocations than in Java. The fact that malloc is slightly slower does not matter if you invoked it 10x less frequently.

kaba0 4y ago
Your point on low-latency GCs is indeed fair (read barriers vs write barriers), and I should have probably specified ref counting where the overhead is much more apparent (and is a fairer comparison). But regarding malloc-only, fragmentation also comes into picture which does have a non-negligable effect. And while Java does like to “heap”-allocate, it happens foremost on thread-local buffers and are used pretty much as an arena allocator. Even without escape analysis, these are very cheap all around.
- pkolaczk 4y ago
  > And while Java does like to “heap”-allocate, it happens foremost on thread-local buffers and are used pretty much as an arena allocator. Even without escape analysis, these are very cheap all around. If that was true, it would be fairly easy for Java to come close to C++, C, Rust, Pascal in the "binary trees" microbenchmark in The Computer Language Benchmarks Game. This microbenchmark is the one that stresses dynamic heap allocation, and is traditionally favoring bump-allocation, so compacting GCs should have an easy win, shouldn't they? The problem is: the best Java implementation loses this benchmark by far on the CPU part (~2.5x worse) and terribly on the memory part (~20x worse) when using the default stop-the-world GC. I attempted to run this benchmark using ZGC with OpenJDK 17, and here are the run times depending on the heap size: 250 MB: OOM 300 MB: 27.3 s 500 MB: 12.4 s 750 MB: 8.4 s 1G: 7.8 s 2G: 5.1 s 16G: 5.6 s For comparison, the best Rust program on the same computer runs in 0.8 seconds and takes about 150-280 MB of RAM (max RSS, varies from run to run). And the C version #5 that uses malloc (no arenas) is still about the same speed as Java at 2G: 5.4 seconds.
  - kaba0 4y ago
    The binary tree benchmark is made to stress test the GC algorithm, and Java beats out every single managed language by a huge degree. Low-level languages don’t do the same thing for this test, so I don’t really see the point of comparing C vs Java on this test - unless using raw memory buffers would be allowed for java as well.
    - pkolaczk 4y ago
      In a comment above you've said Java memory allocation had performance of arena allocators. This benchmark shows such statement is very far from thruth. It is not even close. The best you can shoot for with Java ZGC is the overhead level of naive C malloc/free (not arena) and only if you give it 5x more heap. You can do better only if you are ok with switching to a GC that does STW, in addition to overblown heap. But even with parallel, STW GC Java is still far from from arena allocators. Which is what I have said in the first comment about picking one feature: throughput, low pauses or memory eficiency.
      - kaba0 4y ago
        I didn’t disagree with your last sentence, but nor is the benchmark evidence against “Java memory allocation having the performance of arena allocators”. The benchmark hardcodes additional information about the problem not available/encodeable in Java according to the rules, so Java has to decide at runtime the lifecycle of each object. That of course have an overhead, but it is not allocation rate, or deallocation rate, as both of those are just pointer bump and reuse region, with the surviving objects copied to another region.
        pkolaczk 4y ago
        > That of course have an overhead, but it is not allocation rate, or deallocation rate, as both of those are just pointer bump and reuse region Sure. GC makes some particular operations faster at the expense of adding overhead in a few other places (thrashing the caches by scanning the heap, using precious memory bandwidth to move stuff around, pausing threads of the app from time to time). However, from the developer and end-user perspective it does not matter that allocation is a pointer-bump and deallocation is a no-op. What matters is the performance of all the things together. If there is a performance problem caused by heap allocation, I find it much easier to locate it in a traditional program using manual memory management under-the hood, than in a managed app with tracing GC. This is because the cost is directly associated with the code doing the allocations / deallocations. In managed app, the cost is spread out through unrelated code.

kaba0 4y ago

Your point on low-latency GCs is indeed fair (read barriers vs write barriers), and I should have probably specified ref counting where the overhead is much more apparent (and is a fairer comparison). But regarding malloc-only, fragmentation also comes into picture which does have a non-negligable effect. And while Java does like to “heap”-allocate, it happens foremost on thread-local buffers and are used pretty much as an arena allocator. Even without escape analysis, these are very cheap all around.

pkolaczk 4y ago
> And while Java does like to “heap”-allocate, it happens foremost on thread-local buffers and are used pretty much as an arena allocator. Even without escape analysis, these are very cheap all around. If that was true, it would be fairly easy for Java to come close to C++, C, Rust, Pascal in the "binary trees" microbenchmark in The Computer Language Benchmarks Game. This microbenchmark is the one that stresses dynamic heap allocation, and is traditionally favoring bump-allocation, so compacting GCs should have an easy win, shouldn't they? The problem is: the best Java implementation loses this benchmark by far on the CPU part (~2.5x worse) and terribly on the memory part (~20x worse) when using the default stop-the-world GC. I attempted to run this benchmark using ZGC with OpenJDK 17, and here are the run times depending on the heap size: 250 MB: OOM 300 MB: 27.3 s 500 MB: 12.4 s 750 MB: 8.4 s 1G: 7.8 s 2G: 5.1 s 16G: 5.6 s For comparison, the best Rust program on the same computer runs in 0.8 seconds and takes about 150-280 MB of RAM (max RSS, varies from run to run). And the C version #5 that uses malloc (no arenas) is still about the same speed as Java at 2G: 5.4 seconds.
- kaba0 4y ago
  The binary tree benchmark is made to stress test the GC algorithm, and Java beats out every single managed language by a huge degree. Low-level languages don’t do the same thing for this test, so I don’t really see the point of comparing C vs Java on this test - unless using raw memory buffers would be allowed for java as well.
  - pkolaczk 4y ago
    In a comment above you've said Java memory allocation had performance of arena allocators. This benchmark shows such statement is very far from thruth. It is not even close. The best you can shoot for with Java ZGC is the overhead level of naive C malloc/free (not arena) and only if you give it 5x more heap. You can do better only if you are ok with switching to a GC that does STW, in addition to overblown heap. But even with parallel, STW GC Java is still far from from arena allocators. Which is what I have said in the first comment about picking one feature: throughput, low pauses or memory eficiency.
    - kaba0 4y ago
      I didn’t disagree with your last sentence, but nor is the benchmark evidence against “Java memory allocation having the performance of arena allocators”. The benchmark hardcodes additional information about the problem not available/encodeable in Java according to the rules, so Java has to decide at runtime the lifecycle of each object. That of course have an overhead, but it is not allocation rate, or deallocation rate, as both of those are just pointer bump and reuse region, with the surviving objects copied to another region.
      - pkolaczk 4y ago
        > That of course have an overhead, but it is not allocation rate, or deallocation rate, as both of those are just pointer bump and reuse region Sure. GC makes some particular operations faster at the expense of adding overhead in a few other places (thrashing the caches by scanning the heap, using precious memory bandwidth to move stuff around, pausing threads of the app from time to time). However, from the developer and end-user perspective it does not matter that allocation is a pointer-bump and deallocation is a no-op. What matters is the performance of all the things together. If there is a performance problem caused by heap allocation, I find it much easier to locate it in a traditional program using manual memory management under-the hood, than in a managed app with tracing GC. This is because the cost is directly associated with the code doing the allocations / deallocations. In managed app, the cost is spread out through unrelated code.