15 ms·

Epsilon: The JDK’s Do-Nothing Garbage Collector

141 points by based2 7y ago

pjc50 7y ago
See also Raymond Chen: https://devblogs.microsoft.com/oldnewthing/20180228-00/?p=98125 https://devblogs.microsoft.com/oldnewthing/20180228-00/?p=98... - who quotes the story of a piece of embedded software which accepted leaking since it would be used in the runtime of a missile.
- Aperocky 7y ago
  Sounds like the maintenance shut downs that window OS would need from time to time.
- MaxBarraclough 7y ago
  Similarly, the cooling solution for some missiles' control-systems' is simply 'thermal inertia', i.e. they just race against overheating, as program-termination can be relied upon to occur first.
- matheusmoreira 7y ago
  If the amount of memory needed can be calculated ahead of time, shouldn't it also be possible to allocate it statically?
capableweb 7y ago
Now I'm no Java expert, far from it, so would appreciate any answers to this. I'm interacting with bunch of CLIs that are either in Java or using the JVM otherwise (Clojure mostly), how much of the startup time for this things can be attributed to the GC? It's mentioned in the article that short-running programs (almost all CLIs I use) could use Epsilon since the heap is cleared on exit anyways. But wondering how much of the typical program actually spends on, what I guess is initializing the GC?
- duelingjello 7y ago
  There's an use-case in twelve factor apps where GC pauses would be unacceptable but high availability would allow downtime of an individual stateless app instance. So instead of spending any time GCing, just eat memory and throw it all away and start over fresh as necessary. With various tricks, an instance can be swapped quickly (start a new instance just before killing)... probably want some sort of user-space "OOM killer" to handle it. ulimits lower than JVM option limits would work too, but wouldn't have fast restarts without some magic.
  - capableweb 7y ago
    You might be replying to the wrong comment, or I'm not making my question clear enough. I'm wondering how much of the startup time the GC currently takes, and if using Epsilon will make startup faster.
- lostmyoldone 7y ago
  If you have assigned a JVM too little memory for the short running task it might be a significant amount of time, but if the amount of memory is set correctly the initial GC setup is a fraction of the time compared to the time spent JIT:ing. Although you might gain some performance, I guess this "GC" is going to be used mostly by people running financial - or other - low latency analysis/streaming code where it has been common for years to try to tune the JVM to never even attempt to GC - to avoid latency. The code in these cases are written to reuse most memory, and when the unreclaimable part grows to big, that cluster node stops taking requests - and is then restarted.
- dehrmann 7y ago
  Might want to check out https://github.com/facebook/nailgun https://github.com/facebook/nailgun
- wiradikusuma 7y ago
  If you have access to the source code, you can try to recompile it with Graal Native and make it, well, native executable. It will shave off the load times considerably. But if you have a dozen of them, each of them will have their own JRE embedded so you'll waste disk space
mike_hock 7y ago
To address the short-running program issue, can't the regular GCs just not clean up on program exit?
- amir734jj 7y ago
  That's an interesting concept but unfortunately, garbage collection is not as simple as that.
- capableweb 7y ago
  It's not a issue? It's one of the cases where it does make sense to use Epsilon as the heap is cleared anyway on program exit. From the post: > There is a strong temptation to use Epsilon on deployed programs, rather than to confine it to performance tuning work. As a rule, the Java team discourages this use, with two exceptions. Short-running programs, like all programs, invoke the garbage collector at the end of their run. However, as JEP 318 explains, “accepting the garbage collection cycle to futilely clean up the heap is a waste of time, because the heap would be freed on exit anyway.”
  - mike_hock 7y ago
    Yes, my point is, why do you have to explicitly select the no-op GC for this purpose, when the default GC could already behave this way?
    - chrisseaton 7y ago
      The default GC's do already behave this way. They don't run GC on exit. Why on earth would they?
      - kjeetgill 7y ago
        I believe you are correct but it's worth pointing out that the post above is from an oracle blog and seems to suggest they do run on exit.
- duelingjello 7y ago
walkingolof 7y ago
JVM GC development has seen an upswing in the last few years, one of the most interesting new GCs are ZGC that show impressive numbers, delivers less than 10 ms pause times for any heap size, often than 1 ms for many applications, while having impressive throughput numbers. Another interesting adoption to the containerized world we live in is that GCs recently started to frequently return memory to the OS when not being used. If you want to read more about these new GCs, this is a great post: https://blog.plan99.net/modern-garbage-collection-part-2-1c88847abcfd https://blog.plan99.net/modern-garbage-collection-part-2-1c8...
- kjeetgill 7y ago
  Don't forget Shenandoah headed by G1's lead Christine Flood! I don't know enough to compare it to ZGC but it's an exciting time in Java's GC lineup.
  - lightgreen 7y ago
    They promised low latency in G1, then they promised low latency in Shenandoah, now they promised it in ZGC. Okay. What JVM really really needs for performance is value types: a lot of GC problems will simply not exist with value types. Project Valhalla started five (!) years ago, and they are still working on it. JVM could also use multiple heaps per process (so multiple GCs could run independently not affecting other heap execution and GC). There were talks about it years ago, but nobody is working on it AFAIU.
    - kjeetgill 7y ago
      I always had good experiences with G1 generally. ZGC and Shenandoah are Oracle and Redhat's respective projects with similar goals. I like the options and activies around GC all personally.
    - peterashford 7y ago
      G1 has low latency. And as someone who programs c# for a living: I DO NOT WANT value types.
emmericp 7y ago
Related: I ran some benchmarks for all GCs in OpenJDK 12 some time ago: https://github.com/ixy-languages/ixy-languages/blob/master/Java-garbage-collectors.md https://github.com/ixy-languages/ixy-languages/blob/master/J... Epsilon tied on speed but lost to Shenandoah on latency because never freeing memory isn't ideal either, even if you never run out of memory.
- zamadatix 7y ago
  It'd be interesting to see a version of that graph with just Episilon/Shenandoah. It's hard to tell but it looks like Epsilon may actually have lower average latency but Shenandoah may have lower jitter & max latency.
- CoolGuySteve 7y ago
  Ya if you allocate/deallocate onto a stack, the most recently freed memory is more likely to be hot in the cache for the next allocation, reducing overall latency. It’s something I do all the time in C++.
microcolonel 7y ago
It'd be nice to have a per-thread mode for this.
dmitryminkovsky 7y ago
Probably good for running Java on your cruise missile.
- AndrewBissell 7y ago
  Peter Lawrey said he had HFT clients who wrote Java and managed to get their object allocation so low that they never had a GC pause, and just waited until market close to restart the JVM.
  - aidenn0 7y ago
    A number of games written in managed runtimes will pre-allocate a large number of objects at the start of each level/zone, and hope they don't run out before the next level (where a GC can run).
etaioinshrdlu 7y ago
Speaking of garbage collectors, I had a thought the other day, wondering if performance could have a huge linear speed up if the graph of object pointers was stored compactly in memory, separated from the rest of an application's memory. The object graph could be stored in a succinct compressed format to reduce it's size as much as possible, compared to actual 64 bit pointers. Then the GC algorithm could crawl the graph much much faster by virtue of needing to access far fewer memory pages, and increased likelihood that those pages will be in a fast cpu cache most of the time.
- smaddox 7y ago
  Luajit does something vaguely like this: http://wiki.luajit.org/New-Garbage-Collector#arenas_block-mark-bitmaps http://wiki.luajit.org/New-Garbage-Collector#arenas_block-ma... It doesn't need to access the actual data when sweeping.
  - ufo 7y ago
    The latest version of LuaJIT is still using the old GC. As far as I know the new GC design is in limbo, with no plan to implement it in the near future.
  - tom_mellior 7y ago
    IIUC that only moves the mark bits out of the objects. It still has to access the objects while marking. I think the parent proposed moving pointers out of the objects, so you wouldn't even have to touch the user's objects while marking.
- tom_mellior 7y ago
  How do you envision compressing the graph? One thing you can do, if you know that all your objects will live in one arena, and the maximum size of that arena is <= 2^N objects for some N, is to store "pointers" as N-bit rather than 64-bit values. Java does this for N = 32: https://wiki.openjdk.java.net/display/HotSpot/CompressedOops https://wiki.openjdk.java.net/display/HotSpot/CompressedOops
BLanen 7y ago
I think this article about Instagram turning off Python's garbage collecting is somewhat relevant and interesting: https://instagram-engineering.com/dismissing-python-garbage-collection-at-instagram-4dca40b29172 https://instagram-engineering.com/dismissing-python-garbage-...
- z92 7y ago
  Read it. It's still garbage collecting, but doing it through reference counting. There was an extra GC for circular reference collecting. That one was disabled as well as object freeing up before termination.

141 points by based2 7y ago

See also Raymond Chen: https://devblogs.microsoft.com/oldnewthing/20180228-00/?p=98125 https://devblogs.microsoft.com/oldnewthing/20180228-00/?p=98... - who quotes the story of a piece of embedded software which accepted leaking since it would be used in the runtime of a missile.

Aperocky 7y ago
Sounds like the maintenance shut downs that window OS would need from time to time.
MaxBarraclough 7y ago
Similarly, the cooling solution for some missiles' control-systems' is simply 'thermal inertia', i.e. they just race against overheating, as program-termination can be relied upon to occur first.
matheusmoreira 7y ago
If the amount of memory needed can be calculated ahead of time, shouldn't it also be possible to allocate it statically?

capableweb 7y ago

Now I'm no Java expert, far from it, so would appreciate any answers to this. I'm interacting with bunch of CLIs that are either in Java or using the JVM otherwise (Clojure mostly), how much of the startup time for this things can be attributed to the GC? It's mentioned in the article that short-running programs (almost all CLIs I use) could use Epsilon since the heap is cleared on exit anyways. But wondering how much of the typical program actually spends on, what I guess is initializing the GC?

duelingjello 7y ago
There's an use-case in twelve factor apps where GC pauses would be unacceptable but high availability would allow downtime of an individual stateless app instance. So instead of spending any time GCing, just eat memory and throw it all away and start over fresh as necessary. With various tricks, an instance can be swapped quickly (start a new instance just before killing)... probably want some sort of user-space "OOM killer" to handle it. ulimits lower than JVM option limits would work too, but wouldn't have fast restarts without some magic.
- capableweb 7y ago
  You might be replying to the wrong comment, or I'm not making my question clear enough. I'm wondering how much of the startup time the GC currently takes, and if using Epsilon will make startup faster.
lostmyoldone 7y ago
If you have assigned a JVM too little memory for the short running task it might be a significant amount of time, but if the amount of memory is set correctly the initial GC setup is a fraction of the time compared to the time spent JIT:ing. Although you might gain some performance, I guess this "GC" is going to be used mostly by people running financial - or other - low latency analysis/streaming code where it has been common for years to try to tune the JVM to never even attempt to GC - to avoid latency. The code in these cases are written to reuse most memory, and when the unreclaimable part grows to big, that cluster node stops taking requests - and is then restarted.
dehrmann 7y ago
Might want to check out https://github.com/facebook/nailgun https://github.com/facebook/nailgun
wiradikusuma 7y ago
If you have access to the source code, you can try to recompile it with Graal Native and make it, well, native executable. It will shave off the load times considerably. But if you have a dozen of them, each of them will have their own JRE embedded so you'll waste disk space

mike_hock 7y ago

To address the short-running program issue, can't the regular GCs just not clean up on program exit?

amir734jj 7y ago
That's an interesting concept but unfortunately, garbage collection is not as simple as that.
capableweb 7y ago
It's not a issue? It's one of the cases where it does make sense to use Epsilon as the heap is cleared anyway on program exit. From the post: > There is a strong temptation to use Epsilon on deployed programs, rather than to confine it to performance tuning work. As a rule, the Java team discourages this use, with two exceptions. Short-running programs, like all programs, invoke the garbage collector at the end of their run. However, as JEP 318 explains, “accepting the garbage collection cycle to futilely clean up the heap is a waste of time, because the heap would be freed on exit anyway.”
- mike_hock 7y ago
  Yes, my point is, why do you have to explicitly select the no-op GC for this purpose, when the default GC could already behave this way?
  - chrisseaton 7y ago
    The default GC's do already behave this way. They don't run GC on exit. Why on earth would they?
    - kjeetgill 7y ago
      I believe you are correct but it's worth pointing out that the post above is from an oracle blog and seems to suggest they do run on exit.
duelingjello 7y ago

walkingolof 7y ago

JVM GC development has seen an upswing in the last few years, one of the most interesting new GCs are ZGC that show impressive numbers, delivers less than 10 ms pause times for any heap size, often than 1 ms for many applications, while having impressive throughput numbers. Another interesting adoption to the containerized world we live in is that GCs recently started to frequently return memory to the OS when not being used. If you want to read more about these new GCs, this is a great post: https://blog.plan99.net/modern-garbage-collection-part-2-1c88847abcfd https://blog.plan99.net/modern-garbage-collection-part-2-1c8...

kjeetgill 7y ago
Don't forget Shenandoah headed by G1's lead Christine Flood! I don't know enough to compare it to ZGC but it's an exciting time in Java's GC lineup.
- lightgreen 7y ago
  They promised low latency in G1, then they promised low latency in Shenandoah, now they promised it in ZGC. Okay. What JVM really really needs for performance is value types: a lot of GC problems will simply not exist with value types. Project Valhalla started five (!) years ago, and they are still working on it. JVM could also use multiple heaps per process (so multiple GCs could run independently not affecting other heap execution and GC). There were talks about it years ago, but nobody is working on it AFAIU.
  - kjeetgill 7y ago
    I always had good experiences with G1 generally. ZGC and Shenandoah are Oracle and Redhat's respective projects with similar goals. I like the options and activies around GC all personally.
  - peterashford 7y ago
    G1 has low latency. And as someone who programs c# for a living: I DO NOT WANT value types.

emmericp 7y ago

Related: I ran some benchmarks for all GCs in OpenJDK 12 some time ago: https://github.com/ixy-languages/ixy-languages/blob/master/Java-garbage-collectors.md https://github.com/ixy-languages/ixy-languages/blob/master/J... Epsilon tied on speed but lost to Shenandoah on latency because never freeing memory isn't ideal either, even if you never run out of memory.

zamadatix 7y ago
It'd be interesting to see a version of that graph with just Episilon/Shenandoah. It's hard to tell but it looks like Epsilon may actually have lower average latency but Shenandoah may have lower jitter & max latency.
CoolGuySteve 7y ago
Ya if you allocate/deallocate onto a stack, the most recently freed memory is more likely to be hot in the cache for the next allocation, reducing overall latency. It’s something I do all the time in C++.

microcolonel 7y ago

It'd be nice to have a per-thread mode for this.

dmitryminkovsky 7y ago

Probably good for running Java on your cruise missile.

AndrewBissell 7y ago
Peter Lawrey said he had HFT clients who wrote Java and managed to get their object allocation so low that they never had a GC pause, and just waited until market close to restart the JVM.
- aidenn0 7y ago
  A number of games written in managed runtimes will pre-allocate a large number of objects at the start of each level/zone, and hope they don't run out before the next level (where a GC can run).

etaioinshrdlu 7y ago

Speaking of garbage collectors, I had a thought the other day, wondering if performance could have a huge linear speed up if the graph of object pointers was stored compactly in memory, separated from the rest of an application's memory. The object graph could be stored in a succinct compressed format to reduce it's size as much as possible, compared to actual 64 bit pointers. Then the GC algorithm could crawl the graph much much faster by virtue of needing to access far fewer memory pages, and increased likelihood that those pages will be in a fast cpu cache most of the time.

smaddox 7y ago
Luajit does something vaguely like this: http://wiki.luajit.org/New-Garbage-Collector#arenas_block-mark-bitmaps http://wiki.luajit.org/New-Garbage-Collector#arenas_block-ma... It doesn't need to access the actual data when sweeping.
- ufo 7y ago
  The latest version of LuaJIT is still using the old GC. As far as I know the new GC design is in limbo, with no plan to implement it in the near future.
- tom_mellior 7y ago
  IIUC that only moves the mark bits out of the objects. It still has to access the objects while marking. I think the parent proposed moving pointers out of the objects, so you wouldn't even have to touch the user's objects while marking.
tom_mellior 7y ago
How do you envision compressing the graph? One thing you can do, if you know that all your objects will live in one arena, and the maximum size of that arena is <= 2^N objects for some N, is to store "pointers" as N-bit rather than 64-bit values. Java does this for N = 32: https://wiki.openjdk.java.net/display/HotSpot/CompressedOops https://wiki.openjdk.java.net/display/HotSpot/CompressedOops

BLanen 7y ago

I think this article about Instagram turning off Python's garbage collecting is somewhat relevant and interesting: https://instagram-engineering.com/dismissing-python-garbage-collection-at-instagram-4dca40b29172 https://instagram-engineering.com/dismissing-python-garbage-...

z92 7y ago
Read it. It's still garbage collecting, but doing it through reference counting. There was an extra GC for circular reference collecting. That one was disabled as well as object freeing up before termination.