12 ms·

A very clear and interesting post. I've been trying to fit big-enough long-running stuff into JVMs for a few years, and have found that minimizing the amount o

A very clear and interesting post. I've been trying to fit big-enough long-running stuff into JVMs for a few years, and have found that minimizing the amount of garbage is paramount. Its a bit like games- or C programming. Recent JVM features like 8-bit strings and not having a size-limit on the interned pools etc have been really helpful. But, for my workloads, the big wastes are still things like java.time.Instant and the overhead of temporary strings (which, these days, copy the underlying data. My code worked better when split strings used to just be views). There are collections for much more memory-efficient (and faster) maps and things, and also efficient (and fast) JSON parsing etc. I have evaluated and benchmarked and adopted a few of these kinds of things. Now, when I examine heap-dumps and try and work out where more I can save bytes to keep GC at bay, I mostly see fragments of Instant and String, which are heavily used in my code. If there was only a library that did date manipulation and arithmetic with longs instead of Instant :(

nicktelford 6y ago
> If there was only a library that did date manipulation and arithmetic with longs instead of Instant :( You can always pass around long timestamps and just convert to Instant whenever you need to do any date/time processing. Provided the Instant doesn't escape the method it's allocated in, it should be optimized via inlining and Scalar Replacement so that it doesn't generate garbage. Of course, you'd be adding in the overhead of dividing up your long in to seconds/nanos each time. Note: if this doesn't work on OpenJDK, try GraalVM: it's Partial Escape Analysis should do a better job at finding ways of eliding heap allocations.
- haxen 6y ago
  The worst things happen when value objects are stuck on the heap for a while and then turn into garbage when the value is updated. Escape Analysis doesn't help there, only a good GC can help.
shellac 6y ago
On G1 you used to be able to use `-XX:+UseStringDeduplication`, which gives you back something like the string sharing used pre-whatever it was (8?).
- nicktelford 6y ago
  That appears to still be an option, see the last line of: https://docs.oracle.com/en/java/javase/14/gctuning/garbage-first-g1-garbage-collector1.html#GUID-98E80C82-24D8-41D4-BC39-B2583F04F1FF https://docs.oracle.com/en/java/javase/14/gctuning/garbage-f...
- this_user 6y ago
  If you are looking to do low latency, then G1 isn't the best choice these days, though. Shenandoah or ZGC are both more advanced algorithms that can greatly reduce the pauses caused by GC activity.
  - ec109685 6y ago
    The article compares garbage collectors and with its workloads, Shenandoah had worse throughput, so it depends.
rb808 6y ago
You're absolutely right. Its one reason I struggle with the modern fashion for immutable classes and FP, they are always making copies of everything, seems crazy.
- stu2010 6y ago
  Everything is a trade-off. Immutable objects enables easier safety when having multiple threads do work concurrently. Shared mutable state is still very difficult to do correctly, and at the point where you're introducing locks then you've crippled performance. We have so many cores now that it tends to be a positive trade-off to have many threads doing some wasteful work (copies, extra GC pressure, potentially multiple threads duplicating the same work) than trying to have a perfectly optimized single thread.
  - ThrowawayR2 6y ago
    We have many cores now but memory bandwidth isn't rising as fast. That still makes immutable objects an increasing net loss. It's just a bad idea.
    - entha_saava 6y ago
      Immutability circlejerk is next OOP. It makes sense some times, not always.
- Cthulhu_ 6y ago
  Disclaimer: I'm not a languages expert, but, I think there's a case to make for performance vs clarity / readability. FP is great for parallelisation, multi-core work, things like web servers and other internet-facing services. But probably not the best for number crunching. Horizontal vs vertical scaling, I think. I believe you can express your problem (and solution) better using FP, once you have it solved you can zoom in and replace the most demanding segments with iterative programming, or go down lower to the bare metal.
- logicchains 6y ago
  >Its one reason I struggle with the modern fashion for immutable classes and FP, they are always making copies of everything, seems crazy It depends how it's implemented. It's possible to get very nice performance with immutability and copying through use of an arena allocator, as your stuff will essentially always be in cache (due to reusing the arena), and allocation/deallocation is just bumping a pointer. Of course, not everything easily fits into this approach, but a surprisingly large amount of code can, if designed with it in mind (and using a language that supports it without too much pain, like C/C++). The language Zig is particularly interesting in this regard because everything that allocates takes the allocator as a param, and it has built-in arena allocators in the standard lib.
jcfrei 6y ago
> There are collections for much more memory-efficient (and faster) maps and things, and also efficient (and fast) JSON parsing etc. I have evaluated and benchmarked and adopted a few of these kinds of things. That sounds very interesting. Can you provide links to the benchmarks for fast JSON parsing (libraries)? And the fast maps?
- willvarfar 6y ago
  I can quickly list our findings (from testing on our specific actual workloads; ymmv etc): For collections, we used to use trove but migrated to fastutil a few years ago. For JSON parsing, we are processing lots of very small messages, so use LazyJson. The biggest downside to LazyJson is it doesn't have cheap iteration of keys; the framework could easily provide it. For larger documents, say over a few MB, libraries like Jackson are faster. Yeah, perhaps Java isn't the right tool for our job. And yeah, more recent benchmarking and testing might suggest newer, better libraries than those I have just listed. Its horrific the lengths you have to go to to get good performance Java for the workloads we have; python prototypes run much faster with pypy, and I think that is really about heap management more than code generation. For those of us who know C/C++, its kinda uncomfortable when staring at code and thinking "that temporary string there? 40+ bytes just for the object header!" and things. But, of course, there are advantages to working in memory-safe languages. C# and its structs, yummy.
  - jcfrei 6y ago
    Thanks!
  - laegooose 6y ago
    Wow. I just replaced Java HashMap<Long,Object> with fastutils map for critical cache on a very compute-heavy project of mine, and it instantly got 25% faster. Thank you.
    - chupy 6y ago
      There is another benefit from using the fastutil library, decreased memory usage, which in some cases very significant depending of course on your data structures.
Yeroc 6y ago
There's a saying that "only the good die young" which applies to Java GC. If your Instants and Strings are really short lived then the GC for those is nearly free. For your workload are these objects living on the heap for long enough to be promoted beyond the young generation?

nicktelford 6y ago

> If there was only a library that did date manipulation and arithmetic with longs instead of Instant :( You can always pass around long timestamps and just convert to Instant whenever you need to do any date/time processing. Provided the Instant doesn't escape the method it's allocated in, it should be optimized via inlining and Scalar Replacement so that it doesn't generate garbage. Of course, you'd be adding in the overhead of dividing up your long in to seconds/nanos each time. Note: if this doesn't work on OpenJDK, try GraalVM: it's Partial Escape Analysis should do a better job at finding ways of eliding heap allocations.

haxen 6y ago
The worst things happen when value objects are stuck on the heap for a while and then turn into garbage when the value is updated. Escape Analysis doesn't help there, only a good GC can help.

shellac 6y ago

On G1 you used to be able to use `-XX:+UseStringDeduplication`, which gives you back something like the string sharing used pre-whatever it was (8?).

nicktelford 6y ago
That appears to still be an option, see the last line of: https://docs.oracle.com/en/java/javase/14/gctuning/garbage-first-g1-garbage-collector1.html#GUID-98E80C82-24D8-41D4-BC39-B2583F04F1FF https://docs.oracle.com/en/java/javase/14/gctuning/garbage-f...
this_user 6y ago
If you are looking to do low latency, then G1 isn't the best choice these days, though. Shenandoah or ZGC are both more advanced algorithms that can greatly reduce the pauses caused by GC activity.
- ec109685 6y ago
  The article compares garbage collectors and with its workloads, Shenandoah had worse throughput, so it depends.

rb808 6y ago

You're absolutely right. Its one reason I struggle with the modern fashion for immutable classes and FP, they are always making copies of everything, seems crazy.

stu2010 6y ago
Everything is a trade-off. Immutable objects enables easier safety when having multiple threads do work concurrently. Shared mutable state is still very difficult to do correctly, and at the point where you're introducing locks then you've crippled performance. We have so many cores now that it tends to be a positive trade-off to have many threads doing some wasteful work (copies, extra GC pressure, potentially multiple threads duplicating the same work) than trying to have a perfectly optimized single thread.
- ThrowawayR2 6y ago
  We have many cores now but memory bandwidth isn't rising as fast. That still makes immutable objects an increasing net loss. It's just a bad idea.
  - entha_saava 6y ago
    Immutability circlejerk is next OOP. It makes sense some times, not always.
Cthulhu_ 6y ago
Disclaimer: I'm not a languages expert, but, I think there's a case to make for performance vs clarity / readability. FP is great for parallelisation, multi-core work, things like web servers and other internet-facing services. But probably not the best for number crunching. Horizontal vs vertical scaling, I think. I believe you can express your problem (and solution) better using FP, once you have it solved you can zoom in and replace the most demanding segments with iterative programming, or go down lower to the bare metal.
logicchains 6y ago
>Its one reason I struggle with the modern fashion for immutable classes and FP, they are always making copies of everything, seems crazy It depends how it's implemented. It's possible to get very nice performance with immutability and copying through use of an arena allocator, as your stuff will essentially always be in cache (due to reusing the arena), and allocation/deallocation is just bumping a pointer. Of course, not everything easily fits into this approach, but a surprisingly large amount of code can, if designed with it in mind (and using a language that supports it without too much pain, like C/C++). The language Zig is particularly interesting in this regard because everything that allocates takes the allocator as a param, and it has built-in arena allocators in the standard lib.

jcfrei 6y ago

> There are collections for much more memory-efficient (and faster) maps and things, and also efficient (and fast) JSON parsing etc. I have evaluated and benchmarked and adopted a few of these kinds of things. That sounds very interesting. Can you provide links to the benchmarks for fast JSON parsing (libraries)? And the fast maps?

willvarfar 6y ago
I can quickly list our findings (from testing on our specific actual workloads; ymmv etc): For collections, we used to use trove but migrated to fastutil a few years ago. For JSON parsing, we are processing lots of very small messages, so use LazyJson. The biggest downside to LazyJson is it doesn't have cheap iteration of keys; the framework could easily provide it. For larger documents, say over a few MB, libraries like Jackson are faster. Yeah, perhaps Java isn't the right tool for our job. And yeah, more recent benchmarking and testing might suggest newer, better libraries than those I have just listed. Its horrific the lengths you have to go to to get good performance Java for the workloads we have; python prototypes run much faster with pypy, and I think that is really about heap management more than code generation. For those of us who know C/C++, its kinda uncomfortable when staring at code and thinking "that temporary string there? 40+ bytes just for the object header!" and things. But, of course, there are advantages to working in memory-safe languages. C# and its structs, yummy.
- jcfrei 6y ago
  Thanks!
- laegooose 6y ago
  Wow. I just replaced Java HashMap<Long,Object> with fastutils map for critical cache on a very compute-heavy project of mine, and it instantly got 25% faster. Thank you.
  - chupy 6y ago
    There is another benefit from using the fastutil library, decreased memory usage, which in some cases very significant depending of course on your data structures.

Yeroc 6y ago

There's a saying that "only the good die young" which applies to Java GC. If your Instants and Strings are really short lived then the GC for those is nearly free. For your workload are these objects living on the heap for long enough to be promoted beyond the young generation?