22 ms·

Java’s new garbage collector promises low pause times on multi-terabyte heaps

372 points by ScottWRobinson 8y ago

lamby 8y ago
> Pointer colouring (Is this the same as pointer tagging?)
- rlmw 8y ago
  I would say that pointer colouring is a type of pointer tagging. AFAIK the colouring in ZGC represents different states. So the memory page mappings rely on the pointer having one colour (1 of the 4 bits set) whereas pointer tagging is the term for the general concept of encoding information in pointers.
nicktelford 8y ago
> GC’s SPECjbb 2015 throughput numbers are roughly comparable with the Parallel GC (which optimises for throughput) but with an average pause time of 1ms and a max of 4ms. This is in contrast to G1 and Parallel who had average pause times in excess of 200ms. Not bad! Looking forward to seeing how this performs with a diverse range of workloads as it matures.
- earenndil 8y ago
  I wonder if that would make java more suitable to game development since pauses need to be <8ms, ideally.
  - mysterydip 8y ago
    For a subset of developers, definitely. Overall I don't think it would lead to significantly more adoption, as the main draw (write once deploy everywhere) is already accomplished by engines like Unity.
  - pjmlp 8y ago
    Java is already suitable for game development, specially with LibGDX. Not everyone needs to try to write the next Crisis in it. It certainly has higher performace than all those HTML 5 and Flash games. The biggest problem is that jMonkey is the only RAD tooling available and no match against any of engines that make use of C#.
    - jlg23 8y ago
      > Java is already suitable for game development, specially with LibGDX. I think it is safe to assume that under "game development" most people understand rather high performance demands, not a solitaire clone (esp since the comment you responded to pointed out pause-times as crucial).
      - gameswithgo 8y ago
        You can also make a 3d game of large scale, and just let it have hiccups occasionally and still make a billion dollars. Of course those of us hosting servers for that particular game would have saved a ton of money and pain if they had not used Java, or used Java more cleverly.
        thomasz 8y ago
        Allocating every 3d coordinate on the heap probably wasn't a good idea.
        pjmlp 8y ago
        The point is that they should have used Java more cleverly. One can make a game in C++ and make it just as bad.
        gameswithgo 8y ago
        The lack of value types makes the code quite unwieldy to control memory layout vs C++, or C#, or Go, or Rust or anything
  - nicktelford 8y ago
    I think a lack of value types also hurts Java significantly here. Though I believe they're also on the roadmap.
    - shanxS 8y ago
      Can you please elaborate?
      - pkulak 8y ago
        It's just an object that lives on the stack, not the heap. In lots of languages when you create an object, it's pretty explicit where you're putting it. When Java was created, they wanted to get rid of that complexity, so primitives always go on the stack and objects always* go on the heap. * I think Java has escape analysis, which means that if it can determine that an object you created doesn't leave its context, it will go on the stack and won't add work for the garbage collector.
        chrisseaton 8y ago
        You mention it in your footnote, but objects don't necessarily live on the heap - they can also live on the stack, or in registers, or in vector pipelines, or not at all. Also an interesting aside is that objects are never 'allocated on the stack'. As in you won't find the same layout of memory on the stack as you do on the heap. Instead objects are turned into data edges in the compiler's graph. It's much more abstract than literally doing an alloca.
        pgwhalen 8y ago
        For further details on how objects are not exactly allocated on the stack: https://shipilev.net/jvm-anatomy-park/18-scalar-replacement/ https://shipilev.net/jvm-anatomy-park/18-scalar-replacement/
        electrum 8y ago
        It's more about the memory layout than where the objects live. For example, consider a Point class with (x, y) members. If you have an array of these, each element in the array will be a reference/pointer to a separately allocated Point instance which can be anywhere in the heap. Accessing the array is expensive due to lack of locality (arbitrary memory access). Additionally, each instance has substantial object overhead (typically 8 or 16 bytes). There are various ways to deal with this. One way is to have separate arrays for x and y. Another way is to use a byte array and use Unsafe to read/write the integers. This is terrible from a developer standpoint and is only acceptable if it is a small part of your application. Value types solve this by giving you an efficient memory layout while still allowing you to code against them as normal. The slogan is "codes like a class, works like an int".
  - HohPum1l 8y ago
    On smaller heaps it was already possible to get 5-10ms pauses with the (now deprecated) CMS collector. Unlike Z it was a non-compacting collector though and had some ugly worst-case behaviors.
    - ADefenestrator 8y ago
      Yeah, CMS is great until it isn't, and then it ruins your whole day. A bit of a black art to get tuned well, too, and when your workload changes slightly your tuning is no longer effective and the world stops for 30+ seconds. Hopefully ZGC will be more like G1 in terms of "feed it some extra memory and set a latency target" being all the tuning you need most of the time.
  - phakding 8y ago
    I worked for a company where we had a 100-150 microseconds order latency. We had to tune jvm to extremes. But it is very doable to optimize Java gc for low gc pause times.
    - snaky 8y ago
      If you used to tune JVM only, that might be interesting. Most often the Java code is what being tuned - to the point where there is no point to use Java - with (almost) no gc, off-the-heap memory allocations and all that jazz.
      - dnomad 8y ago
        This doesn't make much sense. Using object pools and and off-heap data structures doesn't warp the code that much, if at all. You still end up with pretty much all clean, simple Java code. I've seen (10+ years old) low-latency Java apps where people avoid using any OO (ie everything is static methods)... and even then the code was still often easier to maintain and understand than equivalent C++ applications. Nobody does this any more anyways.
      - branchless 8y ago
        There are more java developers you can educate to do this than good c++ dev. Plus the tooling is better in java.
      - phakding 8y ago
        It's not either/or. Everything needs to be tuned to achieve the final latency requirements. This includes tuning java code to eliminate creating "garbage", immutable string objects mostly, unnecessary object creation, creating short lived objects compared to long lived etc. The other part right sizing jvm generational partitions, using right gc algo etc. Next part would be to find out the code path with lowest latency requirement and making it hot/jitting. Even next part would be to tune OS for things like thread pinning etc etc. You can go further and tune the network layer. Every little thing counts. You can go even further and employ what's called mechanical sympathy (https://mechanical-sympathy.blogspot.com/ https://mechanical-sympathy.blogspot.com/) and write your code in a way that makes it easier for the processor to process faster. I've never worked with application that use off-heap allocations or no-gc requirements.
  - VikingCoder 8y ago
    Hrm, I would think that if you were developing a game in a GC environment, where GC timing turned out to be a problem, you would get rid of dynamic memory allocation, and try to do everything much more like embedded system programming with fixed memory blocks...? I suppose at the complexity of modern games that's simply not possible anymore...
    - gameswithgo 8y ago
      You can do that, and people do. However a GC language doesn't normally give you the flexibility of getting a block of memory and doing whatever you want inside it, instead you use things like object pools.
      - pjmlp 8y ago
        System.Runtime.InteropServices.Marshal in C#'s case.
        louthy 8y ago
        Also `fixed` in C# [1] Not used Span<T> and Memory<T> [2] in anger yet, but they also seem to be attacking the issue from the other end [1] https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/keywords/fixed-statement https://docs.microsoft.com/en-us/dotnet/csharp/language-refe... [2] https://blogs.msdn.microsoft.com/mazhou/2018/03/25/c-7-series-part-10-spant-and-universal-memory-management/ https://blogs.msdn.microsoft.com/mazhou/2018/03/25/c-7-serie...
      - VikingCoder 8y ago
        I consider object pools to be well within that family of problem solving that I'm talking about.
      - jakewins 8y ago
        You can allocate a block of memory with `ByteBuffer.allocate()` or, if you want it outside the JVM heap, `ByteBuffer.allocateDirect()`. You lose all the object/memory management features, though there are some projects around to provide struct-like APIs: https://github.com/alaisi/nalloc https://github.com/alaisi/nalloc
  - jblow 8y ago
    8ms? Where do you get that number from? If your player has a 144Hz monitor then your pause time, rounded to the nearest millisecond, has to be 0ms.
    - vilhelm_s 8y ago
      Where do you get your number from? 1/(144 Hz) is 7ms. 1/(60 Hz) is 17ms.
      - jblow 8y ago
        I get my number from decades of professional game programming. In a 7ms frame, you are spending most of that frame doing the work of rendering the actual frame (unless your game is so trivial that the GC is going to be easy / fast anyway). An additional millisecond is going to cause you to miss your deadline and drop a frame. Dropped frames feel really bad.
        Scoping_dindus 8y ago
        I nearly spit my drink out from that dudes response, because I just happened to peek at your username as I was reading. It's a crazy place here on HN.
        gravypod 8y ago
        This sort of stalling is only caused by extremely short lived objects. Pooling and provisioning your data ahead of time goes a long way to fixing these specific issues to the degree that they become unnoticeable. There's a lot of tooling built into LWJGL to support this along with their own vector classes which are easily reclaimed. Also has it really been a decade since Braid? Wow.
        jblow 8y ago
        This is true, but it mainly just mitigates the problem. It can never solve the problem, because the whole concept of this kind of scheme is that the memory manager has some volition of its own ... thus it can choose to do things when you don't want or expect (actually this is unavoidable). Also note that this category of answer is basically saying, "look, if you mostly manage your own memory, then GC takes less time!" That's true, but a large part of the value proposition of GC in the first place was to remove the burden of memory management. Once you are saying actually, GC won't do that for this class of application, then really what you are getting out of GC is memory safety (provided the rest of the language is memory-safe). On the one hand, hey, memory-safety is a benefit. On the other hand, I don't think very many people in game development would trade that much performance just for memory safety. (And in fact in game development we very often have to do unsafe memory things. So really what ends up being said is "much of the system has memory safety" which, really, does not sound very alluring.)
- deleted 8y ago
  [deleted]
waterbear 8y ago
Isn’t Java a pay-to-play runtime now? I thought Oracle changed the licensing model after Java 8?
- aseipp 8y ago
  What they did was change the update model for paid versions of the standard Oracle JDK -- you must have a commercial license with Oracle in order to receive security and bugfix updates for copies of the Oracle JDK. OpenJDK, on the other hand, was unaffected by this change; things like security mitigations/fixes land in OpenJDK, and make their way to other versions (like older JDK versions, or Oracle enterprise copies) from there. So, you can go download Zulu or OpenJDK binaries from your distributor (such as your Linux distribution) and you'll be fine (assuming they update promptly). There are also feature distinctions between the Oracle JDK and OpenJDK, but they have been getting smaller over time -- AppCDS and Flight Recorder, which were Oracle-customer-only features, for example, are now in OpenJDK proper (FR will come in JDK 11), and many features such as ZGC (and originally G1, too, I think) were developed in the open and went into OpenJDK directly, right from the start. If you're just running a bog standard Linux/BSD system with some Java software on it, you're almost certainly OpenJDK already anyway, and your distribution maintainers handle security updates for you.
- dig1 8y ago
  That is for Java SE8. Not affected for later releases. Beside, ZGC is part of OpenJDK, which is under GPL.
- koolba 8y ago
  No it’s still free, they (Oracle) are just not supplying the free JVMs. There are plenty of alternatives for OpenJDK builds that are FOSS.
  - antonvs 8y ago
    Oracle still supplies free JVMs. The change was that to get updates for older versions of the JVM, you need to pay.
M_Bakhtiari 8y ago
I'm glad they didn't buy into the refcount meme that's so inexplicably popular these days.
- ahamez 8y ago
  Could you elaborate? Why would refcount be a bad idea ? (genuine question)
  - M_Bakhtiari 8y ago
    Load-increment-store is slow. Load-decrement-branch-if-zero is slow. And you still need a tracing collector for cycles. So it's slow and still nondeterministic.
    - exikyut 8y ago
      Okay, what's better, from a generalist standpoint? I have a decent but very high-level sketched-out understanding of refcounting, but I know nothing else about what's out there, and would be curious to find out. Particularly what would work well on x86 and ARM. An odd and probably completely wrongheaded idea just came to mind: add collectible objects to a lock-free queue in one thread, collect them in another. I wonder if you could do the mark-and-sweep stage across threads too...?
      - pjmlp 8y ago
        Start by reaching out to "The Garbage Collection Handbook" that I mention in another thread.
      - jerven 8y ago
        Better, depends on tradeoff's. RefCounting misses a key feature that ZGC, C4, Shenandoah and other Java GCs do (but not e.g. GO) and that is compacting. Secondly when accessing ref counted objects concurrently ref counting needs to be thread safe. i.e. using atomics, this does have significant performance overheads. Pragmatically in high-throughput situations modern GC's perform better than simple GC. The new things in Java land is that the newest GCs have vastly reduced stop the world times (trending towards OS jitter times), while preserving compaction and without to bad throughput hits. (Even with the misfeatures of finalizers) My personal experience with Shenandoah shows that this changes the way we think about heap settings for java. i.e. just set -Xmx to 60% of machine ram and let idle collections make sure we never use more than 4% in normal days.
obl 8y ago
The memory mapping trick they use on x86 to avoid masking only works up to the maximum 48 bits of addressable virtual memory, so there is much less than 22 free bits in that case. It's also not quite free since it takes up TLB space.
- DSMan195276 8y ago
  Agreed - I'd go farther to say it only works for addresses in the userspace half, so on Linux you lose an extra bit and only get 47 (I don't know how the mappings are setup for Windows/OSX, sorry). It also might be worth pointing out that you now need 16x the page mappings (Since every mapping needs 15 duplicates for the possible flag states) - I don't know how Java does it's memory management but if it does lots of small mappings (Which I'm guessing it does not) then that could be a concern. And like you mentioned with the TLB, it takes up space and it also slows things down a bit since if the flags change the entry in the TLB will no longer be used (since the address is now different) and the MMU will have to walk the page-table again. In practice, I would wager the performance concerns aren't huge, but that's mostly just a guess. I'd personally be interested in seeing a comparison to masking to see just how much slower masking would be, but obviously it's not like they can just flip a switch to use masking.
  - HohPum1l 8y ago
    > It also might be worth pointing out that you now need 16x the page mappings It doesn't. The article mentions it's only 3 mappings since those bits are colors, not arbitrary combinations of flags. > I don't know how Java does it's memory management but if it does lots of small mappings (Which I'm guessing it does not) then that could be a concern. openjdk generally uses large contiguous mappings but it may punch holes in the middle of the heap if it's configured to yield back memory to the OS. But applications that dynamically shrink and expand their heaps are not necessarily those that are concerned about the last quantum of page table overhead. > And like you mentioned with the TLB On the other hand it does support huge pages to mitigate costs of TLB entries.
  - pitaj 8y ago
    > It also might be worth pointing out that you now need 16x the page mappings (Since every mapping needs 15 duplicates for the possible flag states) Nope. They went into this in the article: > Since by design only one of remap, mark0 and mark1 can be 1 at any point in time, it’s possible to do this with three mappings. There’s a nice diagram[1] in the ZGC source for this. [1]: http://hg.openjdk.java.net/zgc/zgc/file/59c07aef65ac/src/hotspot/os_cpu/linux_x86/zGlobals_linux_x86.hpp#l39 http://hg.openjdk.java.net/zgc/zgc/file/59c07aef65ac/src/hot... That might not be totally current, as it doesn't cover the finalizable flag, but if it works the same, that would only be four mappings. If it works differently, then it would be a maximum of 6 mappings. Not 16.
fefe23 8y ago
The site pops up a modal JS popup asking for your e-mail address after a bit of scrolling. That's an instant tab close and blacklist site for me. Why the hell are people doing that to their readers and themselves?
- sj4nz 8y ago
  How do you blacklist the site? Because I'm thinking that would be an interesting extension or browser feature of an anti-bookmark.
igravious 8y ago
4Tb ought to be enough for anybody.
- felipemnoa 8y ago
  Reminds me of the quote "640K ought to be enough for anybody." (I'm guessing you were think of the same [Which apparently is just apocryphal])
- jopsen 8y ago
  I can't imagine many use cases where you wouldn't want to use multiple processes before you hit 4TB. But yeah, I'm sure there are some...
  - M_Bakhtiari 8y ago
    Why would you want to introduce the extra complexity of IPC just because your GC sucks and can't handle the heap size that you need? Besides, processes are a ham fisted way to get around the PDP-11's limited address space and shouldn't be a thing in 2018 anyway.
cridenour 8y ago
All I could think during reading - is this going to affect my Minecraft server parameters?
stuff4ben 8y ago
Interested to know what workloads are using multi-TB heap sizes? Largest I've used has only been in the 48-64GB size and even then it seemed like a waste.
- pjmlp 8y ago
  I have only heard about it in the context of Fintech, where they keep all transaction data around. At least they seem to be Azul´s biggest customer base.
- kjeetgill 8y ago
  1.2 TB Xmx (Java heap) here, reporting in from LinkedIn not fintech. Basically, graphs are hard to shard ... what if we didn't bother? You can query an in memory graph pretty quickly out into the 2nd or 3rd degree. Even G1 (untuned) only pauses for 250ms once every few minuets. It's not perfect but good enough to ignore.
  - hermitdev 8y ago
    250ms is atrocious for finance. Need a small handful of microseconds to not be noticed.
    - kjeetgill 8y ago
      Of course. But this is with zero tuning in place. This is just what the default G1 gets you on that heap size. I'm sure with 2 or 3 flags we could bring that down easily. Given how infrequently we're seeing them (one every few minutes) it's fine for what we're doing.
      - repolfx 8y ago
        Well G1's default pause time goal is 100msec so it won't even try to do better than that without 'tuning' (telling it to pause for less time).
amelius 8y ago
Nice. But unfortunately, it's owned by Oracle. Use it, and you'll be constantly worrying what their lawyers are up to next.
- kevinherron 8y ago
  ZGC is part of OpenJDK and licensed under GPL.
  - amelius 8y ago
    Doesn't matter. Our legal system is such that Oracle can still burn you down, even if you are right.
    - shmageggy 8y ago
      How so in the case of using GPL code?
      - spsoto 8y ago
        I'm not completely sure because I'm not familiar with the topic, but maybe it has to do with the fact that even when they sue you with an impossible-to-win lawsuit, they can destroy you with the cash you need to expend to even start defending yourself. This can also destroy investment rounds for many startups.
        freeone3000 8y ago
        Okay but they can do that even if you don't use Java.
        jerven 8y ago
        And even if you don't do any bussiness with them. Oracle can sue anyone, they sue a lot. Like most companies they only do so if they think they can make a profit. They could sue you, with a higher chance of winning, for using a python, a js or a ruby than for using OpenJDK derived code.
redsymbol 8y ago
OT: I initially mis-read this as "Jay Z's new garbage collector". Talk about an intriguing headline.
exabrial 8y ago
> Limited to 4tb heaps I can't believe the world we're living in, this is incredible. My first computer 512k of ram.
- dboreham 8y ago
  512 bytes here.
  - Cyph0n 8y ago
    What kind of PC had 512 bytes of RAM?
    - gwbas1c 8y ago
      Probably something like: https://en.wikipedia.org/wiki/Altair_8800 https://en.wikipedia.org/wiki/Altair_8800
    - alyandon 8y ago
      Not sure about the parent but my family's first computer had a whopping 4k of ram.
    - monocasa 8y ago
      BASIC on the Atari 2600 did a remarkable amount with 128 bytes of RAM.
    - dboreham 8y ago
      Kim-1, MK-14, other similar systems I built myself. I remember (a couple years later) at age 17 getting a summer job at a POS terminal company where we just took delivery of the first 64kbit DRAMs and being super excited when my boss allowed me to take some of my wages in ram chips at their high volume price..
  - deleted 8y ago
    [deleted]
- userbinator 8y ago
  Java is at least somewhat responsible for the need to get to 4TB in the first place.
Tarean 8y ago
There is also an interesting trick added for Shenandoah to make thread synchronization time bounded. Basically, you don't want to check for gc in each (allocation free) loop iteration. On top of the overhead it makes optimization harder. But if you have 32+ threads then some are probably stuck in loops for a good while which means spinlocks for everyone waiting for the stop-the-world gc. The trick to avoid this is to add an inner loop that does a fixed number of iterations and then do a gc check between each inner loop https://bugs.openjdk.java.net/browse/JDK-8186027 https://bugs.openjdk.java.net/browse/JDK-8186027
schmichael 8y ago
> today AWS will happily rent you an x1e.32xlarge with 128 vCPUs and an incredible 3,904GB of ram. > ... > ZGC restricts itself to 4Tb heaps Isn't a 4TB heap limit short-sighted for a GC intended to last over a decade?
- phoe-krk 8y ago
  4TB uses 42 bits of address space. It leaves you with 22 bits, out of which ZGC uses 4. If anything, borrow a few more bits, and you should easily be able to address petabytes.
  - chrisseaton 8y ago
    > 4TB uses 42 bits of address space. Can't you compress that into less than 42 bits?
    - negativegate 8y ago
      No, compression only works when you don't need to represent every possibility, or can use less bytes for some inputs and more bytes for others. https://en.wikipedia.org/wiki/Pigeonhole_principle#Uses_and_applications https://en.wikipedia.org/wiki/Pigeonhole_principle#Uses_and_...
      - chrisseaton 8y ago
        Why do you need to represent every possibility? You obviously aren't going to squeeze 2^42 objects into 2^42 bytes are you? They each take more than a byte. You don't need to address bytes individually. There's more holes than pigeons here.
        AaronFriel 8y ago
        Suppose we want to store 2^(48-n) 2^n byte objects in your hypothetical <42 bits of address space. Say, 2^47 two byte objects. (Alternatively, more than 2^(48-n) 2^n byte objects in 42 bits of address space.) How do I compute the hash of every object, since by definition I can't address every byte?
foobarbazetc 8y ago
We use Shenandoah everywhere. It’s amazingly great.
- kjeetgill 8y ago
  Is that checked into OpenJDK for the 11 release? Does it have any advantages to ZGC?
  - aseipp 8y ago
    No, it's still a separate fork for Java 8/10/11 as of right now, although some distributions like Red Hat/Fedora do ship with it enabled as a "Technology Preview" on all their enabled JDKs, so you can just get it out of the box. I think Gentoo has it as well. (If it also means anything, I can confirm that enabling Shenandoah is as simple as taking the corresponding OpenJDK code and replacing the `hotspot` subcomponent with the (compatible) Shenandoah fork'd hotspot component, then just building everything normally.)
ryanobjc 8y ago
An interesting fact is that the hotspot team went back on their 20-year conviction that load-barriers reduce performance too much. I guess improvements in CPU and VM designs has changed their minds. This is a big deal, and a major departure. I'm hoping this will improve things, this new architecture more closely resembles Azul's C4. Java and large heaps with non-conforming object allocation patterns (medium aged objects are the most troublesome) sucks really badly.
- repolfx 8y ago
  They do reduce performance a lot. I think what changed is there are now classes of customers who are willing to take a huge performance hit for reliably low pause times or reliably huge heaps. Whereas in the past heaps were smaller and perhaps latency tails were something people cared about less.
- ADefenestrator 8y ago
  G1 does a lot better here in my (somewhat limited) experience. There's some overhead, and you have to give it some extra space, but the mixed collections are decent at making their way through medium-aged objects that survived to the old generation. Still nowhere close to 5-10ms though, and with a fair bit of memory bandwidth burned copying it through the survivor spaces.
quotemstr 8y ago
"For sale: baby shoes, never worn" has nothing on "low pause times on multi-terabyte heaps"
AboutTheWhisles 8y ago
Who are these people that are designing systems that have terabytes of heap under a single GC, but still want/expect/need low pause times. I would think if there were processes that needed consistent latency you would isolate them.
- Inityx 8y ago
  Probably really old teams that had their scope change over time, with decades of technical debt, whose large controlling entities are too slow-moving to afford any kind of large refactor or rewrite in a more suitable language.
- Aweorih 8y ago
  I think I read somewhere, that the heaps of ecommerce website applications can be very huge, because of the loaded strings. I think that was in the time of the java update (9?) where reduced string size was added
openasocket 8y ago
I'm not entirely sure what the point of the multi-mapping is. It says that it means you don't have to do any pointer bitmasking to dereference, but you already have load barriers, which have to at least check to see what phase you are in and possibly do work every time you dereference a pointer, so how much extra latency would doing the bitmask really add? And the problem with the multi-mapping is you will be resolving more virtual addresses, increasing the working set for the TLB. Seems like there could be increased TLB thrashing.
silverlake 8y ago
> Some platforms like SPARC have built-in hardware support for pointer masking so it’s not an issue but for x86, the ZGC team use a neat multi-mapping trick. Does anyone know what this instruction on SPARC is?
comiskey1905 8y ago
How much portion of these comments do consist of Reddit users? Or anyone who's remotely relate with that criminal organization? LOL Fuck off Reddit. FUCK OFF.

> Pointer colouring (Is this the same as pointer tagging?)

rlmw 8y ago
I would say that pointer colouring is a type of pointer tagging. AFAIK the colouring in ZGC represents different states. So the memory page mappings rely on the pointer having one colour (1 of the 4 bits set) whereas pointer tagging is the term for the general concept of encoding information in pointers.

> GC’s SPECjbb 2015 throughput numbers are roughly comparable with the Parallel GC (which optimises for throughput) but with an average pause time of 1ms and a max of 4ms. This is in contrast to G1 and Parallel who had average pause times in excess of 200ms. Not bad! Looking forward to seeing how this performs with a diverse range of workloads as it matures.

earenndil 8y ago
I wonder if that would make java more suitable to game development since pauses need to be <8ms, ideally.
- mysterydip 8y ago
  For a subset of developers, definitely. Overall I don't think it would lead to significantly more adoption, as the main draw (write once deploy everywhere) is already accomplished by engines like Unity.
- pjmlp 8y ago
  Java is already suitable for game development, specially with LibGDX. Not everyone needs to try to write the next Crisis in it. It certainly has higher performace than all those HTML 5 and Flash games. The biggest problem is that jMonkey is the only RAD tooling available and no match against any of engines that make use of C#.
  - jlg23 8y ago
    > Java is already suitable for game development, specially with LibGDX. I think it is safe to assume that under "game development" most people understand rather high performance demands, not a solitaire clone (esp since the comment you responded to pointed out pause-times as crucial).
    - gameswithgo 8y ago
      You can also make a 3d game of large scale, and just let it have hiccups occasionally and still make a billion dollars. Of course those of us hosting servers for that particular game would have saved a ton of money and pain if they had not used Java, or used Java more cleverly.
      - thomasz 8y ago
        Allocating every 3d coordinate on the heap probably wasn't a good idea.
      - pjmlp 8y ago
        The point is that they should have used Java more cleverly. One can make a game in C++ and make it just as bad.
        gameswithgo 8y ago
        The lack of value types makes the code quite unwieldy to control memory layout vs C++, or C#, or Go, or Rust or anything
- nicktelford 8y ago
  I think a lack of value types also hurts Java significantly here. Though I believe they're also on the roadmap.
  - shanxS 8y ago
    Can you please elaborate?
    - pkulak 8y ago
      It's just an object that lives on the stack, not the heap. In lots of languages when you create an object, it's pretty explicit where you're putting it. When Java was created, they wanted to get rid of that complexity, so primitives always go on the stack and objects always* go on the heap. * I think Java has escape analysis, which means that if it can determine that an object you created doesn't leave its context, it will go on the stack and won't add work for the garbage collector.
      - chrisseaton 8y ago
        You mention it in your footnote, but objects don't necessarily live on the heap - they can also live on the stack, or in registers, or in vector pipelines, or not at all. Also an interesting aside is that objects are never 'allocated on the stack'. As in you won't find the same layout of memory on the stack as you do on the heap. Instead objects are turned into data edges in the compiler's graph. It's much more abstract than literally doing an alloca.
        pgwhalen 8y ago
        For further details on how objects are not exactly allocated on the stack: https://shipilev.net/jvm-anatomy-park/18-scalar-replacement/ https://shipilev.net/jvm-anatomy-park/18-scalar-replacement/
      - electrum 8y ago
        It's more about the memory layout than where the objects live. For example, consider a Point class with (x, y) members. If you have an array of these, each element in the array will be a reference/pointer to a separately allocated Point instance which can be anywhere in the heap. Accessing the array is expensive due to lack of locality (arbitrary memory access). Additionally, each instance has substantial object overhead (typically 8 or 16 bytes). There are various ways to deal with this. One way is to have separate arrays for x and y. Another way is to use a byte array and use Unsafe to read/write the integers. This is terrible from a developer standpoint and is only acceptable if it is a small part of your application. Value types solve this by giving you an efficient memory layout while still allowing you to code against them as normal. The slogan is "codes like a class, works like an int".
- HohPum1l 8y ago
  On smaller heaps it was already possible to get 5-10ms pauses with the (now deprecated) CMS collector. Unlike Z it was a non-compacting collector though and had some ugly worst-case behaviors.
  - ADefenestrator 8y ago
    Yeah, CMS is great until it isn't, and then it ruins your whole day. A bit of a black art to get tuned well, too, and when your workload changes slightly your tuning is no longer effective and the world stops for 30+ seconds. Hopefully ZGC will be more like G1 in terms of "feed it some extra memory and set a latency target" being all the tuning you need most of the time.
- phakding 8y ago
  I worked for a company where we had a 100-150 microseconds order latency. We had to tune jvm to extremes. But it is very doable to optimize Java gc for low gc pause times.
  - snaky 8y ago
    If you used to tune JVM only, that might be interesting. Most often the Java code is what being tuned - to the point where there is no point to use Java - with (almost) no gc, off-the-heap memory allocations and all that jazz.
    - dnomad 8y ago
      This doesn't make much sense. Using object pools and and off-heap data structures doesn't warp the code that much, if at all. You still end up with pretty much all clean, simple Java code. I've seen (10+ years old) low-latency Java apps where people avoid using any OO (ie everything is static methods)... and even then the code was still often easier to maintain and understand than equivalent C++ applications. Nobody does this any more anyways.
    - branchless 8y ago
      There are more java developers you can educate to do this than good c++ dev. Plus the tooling is better in java.
    - phakding 8y ago
      It's not either/or. Everything needs to be tuned to achieve the final latency requirements. This includes tuning java code to eliminate creating "garbage", immutable string objects mostly, unnecessary object creation, creating short lived objects compared to long lived etc. The other part right sizing jvm generational partitions, using right gc algo etc. Next part would be to find out the code path with lowest latency requirement and making it hot/jitting. Even next part would be to tune OS for things like thread pinning etc etc. You can go further and tune the network layer. Every little thing counts. You can go even further and employ what's called mechanical sympathy (https://mechanical-sympathy.blogspot.com/ https://mechanical-sympathy.blogspot.com/) and write your code in a way that makes it easier for the processor to process faster. I've never worked with application that use off-heap allocations or no-gc requirements.
- VikingCoder 8y ago
  Hrm, I would think that if you were developing a game in a GC environment, where GC timing turned out to be a problem, you would get rid of dynamic memory allocation, and try to do everything much more like embedded system programming with fixed memory blocks...? I suppose at the complexity of modern games that's simply not possible anymore...
  - gameswithgo 8y ago
    You can do that, and people do. However a GC language doesn't normally give you the flexibility of getting a block of memory and doing whatever you want inside it, instead you use things like object pools.
    - pjmlp 8y ago
      System.Runtime.InteropServices.Marshal in C#'s case.
      - louthy 8y ago
        Also `fixed` in C# [1] Not used Span<T> and Memory<T> [2] in anger yet, but they also seem to be attacking the issue from the other end [1] https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/keywords/fixed-statement https://docs.microsoft.com/en-us/dotnet/csharp/language-refe... [2] https://blogs.msdn.microsoft.com/mazhou/2018/03/25/c-7-series-part-10-spant-and-universal-memory-management/ https://blogs.msdn.microsoft.com/mazhou/2018/03/25/c-7-serie...
    - VikingCoder 8y ago
      I consider object pools to be well within that family of problem solving that I'm talking about.
    - jakewins 8y ago
      You can allocate a block of memory with `ByteBuffer.allocate()` or, if you want it outside the JVM heap, `ByteBuffer.allocateDirect()`. You lose all the object/memory management features, though there are some projects around to provide struct-like APIs: https://github.com/alaisi/nalloc https://github.com/alaisi/nalloc
- jblow 8y ago
  8ms? Where do you get that number from? If your player has a 144Hz monitor then your pause time, rounded to the nearest millisecond, has to be 0ms.
  - vilhelm_s 8y ago
    Where do you get your number from? 1/(144 Hz) is 7ms. 1/(60 Hz) is 17ms.
    - jblow 8y ago
      I get my number from decades of professional game programming. In a 7ms frame, you are spending most of that frame doing the work of rendering the actual frame (unless your game is so trivial that the GC is going to be easy / fast anyway). An additional millisecond is going to cause you to miss your deadline and drop a frame. Dropped frames feel really bad.
      - Scoping_dindus 8y ago
        I nearly spit my drink out from that dudes response, because I just happened to peek at your username as I was reading. It's a crazy place here on HN.
      - gravypod 8y ago
        This sort of stalling is only caused by extremely short lived objects. Pooling and provisioning your data ahead of time goes a long way to fixing these specific issues to the degree that they become unnoticeable. There's a lot of tooling built into LWJGL to support this along with their own vector classes which are easily reclaimed. Also has it really been a decade since Braid? Wow.
        jblow 8y ago
        This is true, but it mainly just mitigates the problem. It can never solve the problem, because the whole concept of this kind of scheme is that the memory manager has some volition of its own ... thus it can choose to do things when you don't want or expect (actually this is unavoidable). Also note that this category of answer is basically saying, "look, if you mostly manage your own memory, then GC takes less time!" That's true, but a large part of the value proposition of GC in the first place was to remove the burden of memory management. Once you are saying actually, GC won't do that for this class of application, then really what you are getting out of GC is memory safety (provided the rest of the language is memory-safe). On the one hand, hey, memory-safety is a benefit. On the other hand, I don't think very many people in game development would trade that much performance just for memory safety. (And in fact in game development we very often have to do unsafe memory things. So really what ends up being said is "much of the system has memory safety" which, really, does not sound very alluring.)
deleted 8y ago
[deleted]

waterbear 8y ago

Isn’t Java a pay-to-play runtime now? I thought Oracle changed the licensing model after Java 8?

aseipp 8y ago
What they did was change the update model for paid versions of the standard Oracle JDK -- you must have a commercial license with Oracle in order to receive security and bugfix updates for copies of the Oracle JDK. OpenJDK, on the other hand, was unaffected by this change; things like security mitigations/fixes land in OpenJDK, and make their way to other versions (like older JDK versions, or Oracle enterprise copies) from there. So, you can go download Zulu or OpenJDK binaries from your distributor (such as your Linux distribution) and you'll be fine (assuming they update promptly). There are also feature distinctions between the Oracle JDK and OpenJDK, but they have been getting smaller over time -- AppCDS and Flight Recorder, which were Oracle-customer-only features, for example, are now in OpenJDK proper (FR will come in JDK 11), and many features such as ZGC (and originally G1, too, I think) were developed in the open and went into OpenJDK directly, right from the start. If you're just running a bog standard Linux/BSD system with some Java software on it, you're almost certainly OpenJDK already anyway, and your distribution maintainers handle security updates for you.
dig1 8y ago
That is for Java SE8. Not affected for later releases. Beside, ZGC is part of OpenJDK, which is under GPL.
koolba 8y ago
No it’s still free, they (Oracle) are just not supplying the free JVMs. There are plenty of alternatives for OpenJDK builds that are FOSS.
- antonvs 8y ago
  Oracle still supplies free JVMs. The change was that to get updates for older versions of the JVM, you need to pay.

M_Bakhtiari 8y ago

I'm glad they didn't buy into the refcount meme that's so inexplicably popular these days.

ahamez 8y ago
Could you elaborate? Why would refcount be a bad idea ? (genuine question)
- M_Bakhtiari 8y ago
  Load-increment-store is slow. Load-decrement-branch-if-zero is slow. And you still need a tracing collector for cycles. So it's slow and still nondeterministic.
  - exikyut 8y ago
    Okay, what's better, from a generalist standpoint? I have a decent but very high-level sketched-out understanding of refcounting, but I know nothing else about what's out there, and would be curious to find out. Particularly what would work well on x86 and ARM. An odd and probably completely wrongheaded idea just came to mind: add collectible objects to a lock-free queue in one thread, collect them in another. I wonder if you could do the mark-and-sweep stage across threads too...?
    - pjmlp 8y ago
      Start by reaching out to "The Garbage Collection Handbook" that I mention in another thread.
    - jerven 8y ago
      Better, depends on tradeoff's. RefCounting misses a key feature that ZGC, C4, Shenandoah and other Java GCs do (but not e.g. GO) and that is compacting. Secondly when accessing ref counted objects concurrently ref counting needs to be thread safe. i.e. using atomics, this does have significant performance overheads. Pragmatically in high-throughput situations modern GC's perform better than simple GC. The new things in Java land is that the newest GCs have vastly reduced stop the world times (trending towards OS jitter times), while preserving compaction and without to bad throughput hits. (Even with the misfeatures of finalizers) My personal experience with Shenandoah shows that this changes the way we think about heap settings for java. i.e. just set -Xmx to 60% of machine ram and let idle collections make sure we never use more than 4% in normal days.

obl 8y ago

The memory mapping trick they use on x86 to avoid masking only works up to the maximum 48 bits of addressable virtual memory, so there is much less than 22 free bits in that case. It's also not quite free since it takes up TLB space.

DSMan195276 8y ago
Agreed - I'd go farther to say it only works for addresses in the userspace half, so on Linux you lose an extra bit and only get 47 (I don't know how the mappings are setup for Windows/OSX, sorry). It also might be worth pointing out that you now need 16x the page mappings (Since every mapping needs 15 duplicates for the possible flag states) - I don't know how Java does it's memory management but if it does lots of small mappings (Which I'm guessing it does not) then that could be a concern. And like you mentioned with the TLB, it takes up space and it also slows things down a bit since if the flags change the entry in the TLB will no longer be used (since the address is now different) and the MMU will have to walk the page-table again. In practice, I would wager the performance concerns aren't huge, but that's mostly just a guess. I'd personally be interested in seeing a comparison to masking to see just how much slower masking would be, but obviously it's not like they can just flip a switch to use masking.
- HohPum1l 8y ago
  > It also might be worth pointing out that you now need 16x the page mappings It doesn't. The article mentions it's only 3 mappings since those bits are colors, not arbitrary combinations of flags. > I don't know how Java does it's memory management but if it does lots of small mappings (Which I'm guessing it does not) then that could be a concern. openjdk generally uses large contiguous mappings but it may punch holes in the middle of the heap if it's configured to yield back memory to the OS. But applications that dynamically shrink and expand their heaps are not necessarily those that are concerned about the last quantum of page table overhead. > And like you mentioned with the TLB On the other hand it does support huge pages to mitigate costs of TLB entries.
- pitaj 8y ago
  > It also might be worth pointing out that you now need 16x the page mappings (Since every mapping needs 15 duplicates for the possible flag states) Nope. They went into this in the article: > Since by design only one of remap, mark0 and mark1 can be 1 at any point in time, it’s possible to do this with three mappings. There’s a nice diagram[1] in the ZGC source for this. [1]: http://hg.openjdk.java.net/zgc/zgc/file/59c07aef65ac/src/hotspot/os_cpu/linux_x86/zGlobals_linux_x86.hpp#l39 http://hg.openjdk.java.net/zgc/zgc/file/59c07aef65ac/src/hot... That might not be totally current, as it doesn't cover the finalizable flag, but if it works the same, that would only be four mappings. If it works differently, then it would be a maximum of 6 mappings. Not 16.

fefe23 8y ago

The site pops up a modal JS popup asking for your e-mail address after a bit of scrolling. That's an instant tab close and blacklist site for me. Why the hell are people doing that to their readers and themselves?

sj4nz 8y ago
How do you blacklist the site? Because I'm thinking that would be an interesting extension or browser feature of an anti-bookmark.

igravious 8y ago

4Tb ought to be enough for anybody.

felipemnoa 8y ago
Reminds me of the quote "640K ought to be enough for anybody." (I'm guessing you were think of the same [Which apparently is just apocryphal])
jopsen 8y ago
I can't imagine many use cases where you wouldn't want to use multiple processes before you hit 4TB. But yeah, I'm sure there are some...
- M_Bakhtiari 8y ago
  Why would you want to introduce the extra complexity of IPC just because your GC sucks and can't handle the heap size that you need? Besides, processes are a ham fisted way to get around the PDP-11's limited address space and shouldn't be a thing in 2018 anyway.

cridenour 8y ago

All I could think during reading - is this going to affect my Minecraft server parameters?

stuff4ben 8y ago

Interested to know what workloads are using multi-TB heap sizes? Largest I've used has only been in the 48-64GB size and even then it seemed like a waste.

pjmlp 8y ago
I have only heard about it in the context of Fintech, where they keep all transaction data around. At least they seem to be Azul´s biggest customer base.
kjeetgill 8y ago
1.2 TB Xmx (Java heap) here, reporting in from LinkedIn not fintech. Basically, graphs are hard to shard ... what if we didn't bother? You can query an in memory graph pretty quickly out into the 2nd or 3rd degree. Even G1 (untuned) only pauses for 250ms once every few minuets. It's not perfect but good enough to ignore.
- hermitdev 8y ago
  250ms is atrocious for finance. Need a small handful of microseconds to not be noticed.
  - kjeetgill 8y ago
    Of course. But this is with zero tuning in place. This is just what the default G1 gets you on that heap size. I'm sure with 2 or 3 flags we could bring that down easily. Given how infrequently we're seeing them (one every few minutes) it's fine for what we're doing.
    - repolfx 8y ago
      Well G1's default pause time goal is 100msec so it won't even try to do better than that without 'tuning' (telling it to pause for less time).

amelius 8y ago

Nice. But unfortunately, it's owned by Oracle. Use it, and you'll be constantly worrying what their lawyers are up to next.

kevinherron 8y ago
ZGC is part of OpenJDK and licensed under GPL.
- amelius 8y ago
  Doesn't matter. Our legal system is such that Oracle can still burn you down, even if you are right.
  - shmageggy 8y ago
    How so in the case of using GPL code?
    - spsoto 8y ago
      I'm not completely sure because I'm not familiar with the topic, but maybe it has to do with the fact that even when they sue you with an impossible-to-win lawsuit, they can destroy you with the cash you need to expend to even start defending yourself. This can also destroy investment rounds for many startups.
      - freeone3000 8y ago
        Okay but they can do that even if you don't use Java.
        jerven 8y ago
        And even if you don't do any bussiness with them. Oracle can sue anyone, they sue a lot. Like most companies they only do so if they think they can make a profit. They could sue you, with a higher chance of winning, for using a python, a js or a ruby than for using OpenJDK derived code.

redsymbol 8y ago

OT: I initially mis-read this as "Jay Z's new garbage collector". Talk about an intriguing headline.

exabrial 8y ago

> Limited to 4tb heaps I can't believe the world we're living in, this is incredible. My first computer 512k of ram.

dboreham 8y ago
512 bytes here.
- Cyph0n 8y ago
  What kind of PC had 512 bytes of RAM?
  - gwbas1c 8y ago
    Probably something like: https://en.wikipedia.org/wiki/Altair_8800 https://en.wikipedia.org/wiki/Altair_8800
  - alyandon 8y ago
    Not sure about the parent but my family's first computer had a whopping 4k of ram.
  - monocasa 8y ago
    BASIC on the Atari 2600 did a remarkable amount with 128 bytes of RAM.
  - dboreham 8y ago
    Kim-1, MK-14, other similar systems I built myself. I remember (a couple years later) at age 17 getting a summer job at a POS terminal company where we just took delivery of the first 64kbit DRAMs and being super excited when my boss allowed me to take some of my wages in ram chips at their high volume price..
- deleted 8y ago
  [deleted]
userbinator 8y ago
Java is at least somewhat responsible for the need to get to 4TB in the first place.

Tarean 8y ago

There is also an interesting trick added for Shenandoah to make thread synchronization time bounded. Basically, you don't want to check for gc in each (allocation free) loop iteration. On top of the overhead it makes optimization harder. But if you have 32+ threads then some are probably stuck in loops for a good while which means spinlocks for everyone waiting for the stop-the-world gc. The trick to avoid this is to add an inner loop that does a fixed number of iterations and then do a gc check between each inner loop https://bugs.openjdk.java.net/browse/JDK-8186027 https://bugs.openjdk.java.net/browse/JDK-8186027

schmichael 8y ago

> today AWS will happily rent you an x1e.32xlarge with 128 vCPUs and an incredible 3,904GB of ram. > ... > ZGC restricts itself to 4Tb heaps Isn't a 4TB heap limit short-sighted for a GC intended to last over a decade?

phoe-krk 8y ago
4TB uses 42 bits of address space. It leaves you with 22 bits, out of which ZGC uses 4. If anything, borrow a few more bits, and you should easily be able to address petabytes.
- chrisseaton 8y ago
  > 4TB uses 42 bits of address space. Can't you compress that into less than 42 bits?
  - negativegate 8y ago
    No, compression only works when you don't need to represent every possibility, or can use less bytes for some inputs and more bytes for others. https://en.wikipedia.org/wiki/Pigeonhole_principle#Uses_and_applications https://en.wikipedia.org/wiki/Pigeonhole_principle#Uses_and_...
    - chrisseaton 8y ago
      Why do you need to represent every possibility? You obviously aren't going to squeeze 2^42 objects into 2^42 bytes are you? They each take more than a byte. You don't need to address bytes individually. There's more holes than pigeons here.
      - AaronFriel 8y ago
        Suppose we want to store 2^(48-n) 2^n byte objects in your hypothetical <42 bits of address space. Say, 2^47 two byte objects. (Alternatively, more than 2^(48-n) 2^n byte objects in 42 bits of address space.) How do I compute the hash of every object, since by definition I can't address every byte?

foobarbazetc 8y ago

We use Shenandoah everywhere. It’s amazingly great.

kjeetgill 8y ago
Is that checked into OpenJDK for the 11 release? Does it have any advantages to ZGC?
- aseipp 8y ago
  No, it's still a separate fork for Java 8/10/11 as of right now, although some distributions like Red Hat/Fedora do ship with it enabled as a "Technology Preview" on all their enabled JDKs, so you can just get it out of the box. I think Gentoo has it as well. (If it also means anything, I can confirm that enabling Shenandoah is as simple as taking the corresponding OpenJDK code and replacing the `hotspot` subcomponent with the (compatible) Shenandoah fork'd hotspot component, then just building everything normally.)

ryanobjc 8y ago

An interesting fact is that the hotspot team went back on their 20-year conviction that load-barriers reduce performance too much. I guess improvements in CPU and VM designs has changed their minds. This is a big deal, and a major departure. I'm hoping this will improve things, this new architecture more closely resembles Azul's C4. Java and large heaps with non-conforming object allocation patterns (medium aged objects are the most troublesome) sucks really badly.

repolfx 8y ago
They do reduce performance a lot. I think what changed is there are now classes of customers who are willing to take a huge performance hit for reliably low pause times or reliably huge heaps. Whereas in the past heaps were smaller and perhaps latency tails were something people cared about less.
ADefenestrator 8y ago
G1 does a lot better here in my (somewhat limited) experience. There's some overhead, and you have to give it some extra space, but the mixed collections are decent at making their way through medium-aged objects that survived to the old generation. Still nowhere close to 5-10ms though, and with a fair bit of memory bandwidth burned copying it through the survivor spaces.

quotemstr 8y ago

"For sale: baby shoes, never worn" has nothing on "low pause times on multi-terabyte heaps"

AboutTheWhisles 8y ago

Who are these people that are designing systems that have terabytes of heap under a single GC, but still want/expect/need low pause times. I would think if there were processes that needed consistent latency you would isolate them.

Inityx 8y ago
Probably really old teams that had their scope change over time, with decades of technical debt, whose large controlling entities are too slow-moving to afford any kind of large refactor or rewrite in a more suitable language.
Aweorih 8y ago
I think I read somewhere, that the heaps of ecommerce website applications can be very huge, because of the loaded strings. I think that was in the time of the java update (9?) where reduced string size was added

openasocket 8y ago

I'm not entirely sure what the point of the multi-mapping is. It says that it means you don't have to do any pointer bitmasking to dereference, but you already have load barriers, which have to at least check to see what phase you are in and possibly do work every time you dereference a pointer, so how much extra latency would doing the bitmask really add? And the problem with the multi-mapping is you will be resolving more virtual addresses, increasing the working set for the TLB. Seems like there could be increased TLB thrashing.

silverlake 8y ago

> Some platforms like SPARC have built-in hardware support for pointer masking so it’s not an issue but for x86, the ZGC team use a neat multi-mapping trick. Does anyone know what this instruction on SPARC is?

comiskey1905 8y ago

How much portion of these comments do consist of Reddit users? Or anyone who's remotely relate with that criminal organization? LOL Fuck off Reddit. FUCK OFF.