3 ms·

> And while Java does like to “heap”-allocate, it happens foremost on thread-local buffers and are used pretty much as an arena allocator. Even without escape a

> And while Java does like to “heap”-allocate, it happens foremost on thread-local buffers and are used pretty much as an arena allocator. Even without escape analysis, these are very cheap all around. If that was true, it would be fairly easy for Java to come close to C++, C, Rust, Pascal in the "binary trees" microbenchmark in The Computer Language Benchmarks Game. This microbenchmark is the one that stresses dynamic heap allocation, and is traditionally favoring bump-allocation, so compacting GCs should have an easy win, shouldn't they? The problem is: the best Java implementation loses this benchmark by far on the CPU part (~2.5x worse) and terribly on the memory part (~20x worse) when using the default stop-the-world GC. I attempted to run this benchmark using ZGC with OpenJDK 17, and here are the run times depending on the heap size: 250 MB: OOM 300 MB: 27.3 s 500 MB: 12.4 s 750 MB: 8.4 s 1G: 7.8 s 2G: 5.1 s 16G: 5.6 s For comparison, the best Rust program on the same computer runs in 0.8 seconds and takes about 150-280 MB of RAM (max RSS, varies from run to run). And the C version #5 that uses malloc (no arenas) is still about the same speed as Java at 2G: 5.4 seconds.

kaba0 4y ago
The binary tree benchmark is made to stress test the GC algorithm, and Java beats out every single managed language by a huge degree. Low-level languages don’t do the same thing for this test, so I don’t really see the point of comparing C vs Java on this test - unless using raw memory buffers would be allowed for java as well.
- pkolaczk 4y ago
  In a comment above you've said Java memory allocation had performance of arena allocators. This benchmark shows such statement is very far from thruth. It is not even close. The best you can shoot for with Java ZGC is the overhead level of naive C malloc/free (not arena) and only if you give it 5x more heap. You can do better only if you are ok with switching to a GC that does STW, in addition to overblown heap. But even with parallel, STW GC Java is still far from from arena allocators. Which is what I have said in the first comment about picking one feature: throughput, low pauses or memory eficiency.
  - kaba0 4y ago
    I didn’t disagree with your last sentence, but nor is the benchmark evidence against “Java memory allocation having the performance of arena allocators”. The benchmark hardcodes additional information about the problem not available/encodeable in Java according to the rules, so Java has to decide at runtime the lifecycle of each object. That of course have an overhead, but it is not allocation rate, or deallocation rate, as both of those are just pointer bump and reuse region, with the surviving objects copied to another region.
    - pkolaczk 4y ago
      > That of course have an overhead, but it is not allocation rate, or deallocation rate, as both of those are just pointer bump and reuse region Sure. GC makes some particular operations faster at the expense of adding overhead in a few other places (thrashing the caches by scanning the heap, using precious memory bandwidth to move stuff around, pausing threads of the app from time to time). However, from the developer and end-user perspective it does not matter that allocation is a pointer-bump and deallocation is a no-op. What matters is the performance of all the things together. If there is a performance problem caused by heap allocation, I find it much easier to locate it in a traditional program using manual memory management under-the hood, than in a managed app with tracing GC. This is because the cost is directly associated with the code doing the allocations / deallocations. In managed app, the cost is spread out through unrelated code.

kaba0 4y ago

The binary tree benchmark is made to stress test the GC algorithm, and Java beats out every single managed language by a huge degree. Low-level languages don’t do the same thing for this test, so I don’t really see the point of comparing C vs Java on this test - unless using raw memory buffers would be allowed for java as well.

pkolaczk 4y ago
In a comment above you've said Java memory allocation had performance of arena allocators. This benchmark shows such statement is very far from thruth. It is not even close. The best you can shoot for with Java ZGC is the overhead level of naive C malloc/free (not arena) and only if you give it 5x more heap. You can do better only if you are ok with switching to a GC that does STW, in addition to overblown heap. But even with parallel, STW GC Java is still far from from arena allocators. Which is what I have said in the first comment about picking one feature: throughput, low pauses or memory eficiency.
- kaba0 4y ago
  I didn’t disagree with your last sentence, but nor is the benchmark evidence against “Java memory allocation having the performance of arena allocators”. The benchmark hardcodes additional information about the problem not available/encodeable in Java according to the rules, so Java has to decide at runtime the lifecycle of each object. That of course have an overhead, but it is not allocation rate, or deallocation rate, as both of those are just pointer bump and reuse region, with the surviving objects copied to another region.
  - pkolaczk 4y ago
    > That of course have an overhead, but it is not allocation rate, or deallocation rate, as both of those are just pointer bump and reuse region Sure. GC makes some particular operations faster at the expense of adding overhead in a few other places (thrashing the caches by scanning the heap, using precious memory bandwidth to move stuff around, pausing threads of the app from time to time). However, from the developer and end-user perspective it does not matter that allocation is a pointer-bump and deallocation is a no-op. What matters is the performance of all the things together. If there is a performance problem caused by heap allocation, I find it much easier to locate it in a traditional program using manual memory management under-the hood, than in a managed app with tracing GC. This is because the cost is directly associated with the code doing the allocations / deallocations. In managed app, the cost is spread out through unrelated code.