3 ms·

Max pause times of 0.5ms is what got me really interested. It feels like a huge trade-off of GCs is almost completely gone. https://malloc.se/blog/zgc-jdk16 h

by majou 5y ago

Max pause times of 0.5ms is what got me really interested. It feels like a huge trade-off of GCs is almost completely gone. https://malloc.se/blog/zgc-jdk16 https://malloc.se/blog/zgc-jdk16

silon42 5y ago
There's still a memory tradeoff, some due to GC, some due to Java (lots of runtime reflection...). Guessing 2-4x.
- masklinn 5y ago
  The GC memory overhead affects all languages with a GC more advanced than refcounting. It certainly does affect Go as well.
- kaba0 5y ago
  Also, throughput. But latency and throughput are almost universally opposite ends of the same axis — that’s why it’s great that Java allows for choosing a GC implementation.
  - jatone 5y ago
    you can fix throughput problems by adding compute resources, you can't fix latency issues. I'll always pick a GC that optimizes latency over throughput. its easier to maintain the software.
- marginalia_nu 5y ago
  This is a bit of a tangent, but you can get into situations where Java's memory-overhead becomes pretty untenable. I was in a situation of having to keep track of ~1 billion short strings of a median length of maybe 7 characters. In terms of just data, that should clock in at about 10 Gb; in practice it was closer to 24 Gb. I tried going with just byte[]-instances instead, which didn't help a lot. Using long byte[]-instances and indexing those doesn't help as much as you'd think because they get sliced up into small objects behind the scenes. I ended up memory mapping blocks of memory and basically implementing my own append-only allocator.
  - hashmash 5y ago
    The Lilliput project aims to address this: https://wiki.openjdk.java.net/display/lilliput https://wiki.openjdk.java.net/display/lilliput
  - jeeeb 5y ago
    FWIW. This would probably present a challenge in most (all?) languages. For example in libc++ due to SSO an std::string has a minimum size of 24 bytes. For a billion strings less than 15 chars (+ the null byte) that gets you to 24GB, and that’s optimistically assuming each string is allocated in place. I doubt heap allocated char* would do much better either. Just having a billion 8 byte pointers eats a lot of memory. You’d really need some sort of string packing scheme similar to what you did in Java.
    - marginalia_nu 5y ago
      It's a lot easier to build custom allocators in C++ though. For one, Java has a maximum mmap-size of 2 Gb, and as a cherry on top of that turd, you have no control over their lifecycle. The language is very clearly not designed for this type of work, and if you try to make it do it anyway, it fights you every step of the way.
      - kasperni 5y ago
        The foreign memory API which is currently incubating should help with most of these limitations: https://openjdk.java.net/jeps/419 https://openjdk.java.net/jeps/419
        native_samples 5y ago
        Right - specifically they invented a way to make closing an mmapped segment safe and fast. The reason you can't officially (without private APIs) unmap something in current Java is because if you did then other threads would segfault, and "no segfaults" is kind a defining characteristic of Java. The new API fixes this using more VM magic, so closing a mapping from one thread will cause other threads to throw exceptions, but this is done without sacrificing performance (it doesn't require checking a status every time you read from memory).
masklinn 5y ago
> It feels like a huge trade-off of GCs is almost completely gone. FWIW the tradeoff of low latency GC is usually paid in throughput. That is definitely the case for Go, which can lag very much behind allocations (so if your allocation pattern is bad enough the heap will keep growing despite the live heap being stable, because the GC is unable to clear the dead heap fast enough for the new allocations).
- jatone 5y ago
  throughput can be fixed by adding compute. latency cannot. always optimize for latency with gc. and no the heap will not keep growing in golang. it'll force threads to help with GC if its falling behind. thereby reducing the rate of allocations and speeding up the collection.
  - native_samples 5y ago
    Only in some kinds of apps, like web servers where all the heavy lifting is being done by the database anyway. Consider a compiler. It's not infinitely scalable to multiple cores. It may not even be multi-threaded at all. It also doesn't care about pause times - for that you want Parallel GC.
    - jatone 5y ago
      depends on the compiler and language. golang seems to counter point your position quite handedly. having one of the fastest compile times and being highly concurrent. if your application is so simple it doesn't use concurrency then 99% of the time you can completely remove the need for GC by preallocating slabs.
      - native_samples 5y ago
        The Go compiler is fast because it doesn't do very much, not because Go's GC is good for throughput oriented jobs. Go has repeatedly tied itself in knots over the years because they have a goal of not making the compiler slower, yet, the GC needs of the compiler are diametrically opposed to the needs of the HTTP servers Go is normally used for. https://go.dev/blog/ismmkeynote https://go.dev/blog/ismmkeynote "As you can see if you have ROC on and not a lot of sharing, things actually scale quite nicely. If you don’t have ROC on it wasn’t nearly as good ... At that point there was a lot of concern about our compiler and we could not slow down our compilers. Unfortunately the compilers were exactly the programs that ROC did not do well at. We were seeing 30, 40, 50% and more slowdowns and that was unacceptable. Go is proud of how fast its compiler is."

by majou 5y ago

Max pause times of 0.5ms is what got me really interested. It feels like a huge trade-off of GCs is almost completely gone. https://malloc.se/blog/zgc-jdk16 https://malloc.se/blog/zgc-jdk16

silon42 5y ago

There's still a memory tradeoff, some due to GC, some due to Java (lots of runtime reflection...). Guessing 2-4x.

masklinn 5y ago
The GC memory overhead affects all languages with a GC more advanced than refcounting. It certainly does affect Go as well.
kaba0 5y ago
Also, throughput. But latency and throughput are almost universally opposite ends of the same axis — that’s why it’s great that Java allows for choosing a GC implementation.
- jatone 5y ago
  you can fix throughput problems by adding compute resources, you can't fix latency issues. I'll always pick a GC that optimizes latency over throughput. its easier to maintain the software.
marginalia_nu 5y ago
This is a bit of a tangent, but you can get into situations where Java's memory-overhead becomes pretty untenable. I was in a situation of having to keep track of ~1 billion short strings of a median length of maybe 7 characters. In terms of just data, that should clock in at about 10 Gb; in practice it was closer to 24 Gb. I tried going with just byte[]-instances instead, which didn't help a lot. Using long byte[]-instances and indexing those doesn't help as much as you'd think because they get sliced up into small objects behind the scenes. I ended up memory mapping blocks of memory and basically implementing my own append-only allocator.
- hashmash 5y ago
  The Lilliput project aims to address this: https://wiki.openjdk.java.net/display/lilliput https://wiki.openjdk.java.net/display/lilliput
- jeeeb 5y ago
  FWIW. This would probably present a challenge in most (all?) languages. For example in libc++ due to SSO an std::string has a minimum size of 24 bytes. For a billion strings less than 15 chars (+ the null byte) that gets you to 24GB, and that’s optimistically assuming each string is allocated in place. I doubt heap allocated char* would do much better either. Just having a billion 8 byte pointers eats a lot of memory. You’d really need some sort of string packing scheme similar to what you did in Java.
  - marginalia_nu 5y ago
    It's a lot easier to build custom allocators in C++ though. For one, Java has a maximum mmap-size of 2 Gb, and as a cherry on top of that turd, you have no control over their lifecycle. The language is very clearly not designed for this type of work, and if you try to make it do it anyway, it fights you every step of the way.
    - kasperni 5y ago
      The foreign memory API which is currently incubating should help with most of these limitations: https://openjdk.java.net/jeps/419 https://openjdk.java.net/jeps/419
      - native_samples 5y ago
        Right - specifically they invented a way to make closing an mmapped segment safe and fast. The reason you can't officially (without private APIs) unmap something in current Java is because if you did then other threads would segfault, and "no segfaults" is kind a defining characteristic of Java. The new API fixes this using more VM magic, so closing a mapping from one thread will cause other threads to throw exceptions, but this is done without sacrificing performance (it doesn't require checking a status every time you read from memory).

masklinn 5y ago

> It feels like a huge trade-off of GCs is almost completely gone. FWIW the tradeoff of low latency GC is usually paid in throughput. That is definitely the case for Go, which can lag very much behind allocations (so if your allocation pattern is bad enough the heap will keep growing despite the live heap being stable, because the GC is unable to clear the dead heap fast enough for the new allocations).

jatone 5y ago
throughput can be fixed by adding compute. latency cannot. always optimize for latency with gc. and no the heap will not keep growing in golang. it'll force threads to help with GC if its falling behind. thereby reducing the rate of allocations and speeding up the collection.
- native_samples 5y ago
  Only in some kinds of apps, like web servers where all the heavy lifting is being done by the database anyway. Consider a compiler. It's not infinitely scalable to multiple cores. It may not even be multi-threaded at all. It also doesn't care about pause times - for that you want Parallel GC.
  - jatone 5y ago
    depends on the compiler and language. golang seems to counter point your position quite handedly. having one of the fastest compile times and being highly concurrent. if your application is so simple it doesn't use concurrency then 99% of the time you can completely remove the need for GC by preallocating slabs.
    - native_samples 5y ago
      The Go compiler is fast because it doesn't do very much, not because Go's GC is good for throughput oriented jobs. Go has repeatedly tied itself in knots over the years because they have a goal of not making the compiler slower, yet, the GC needs of the compiler are diametrically opposed to the needs of the HTTP servers Go is normally used for. https://go.dev/blog/ismmkeynote https://go.dev/blog/ismmkeynote "As you can see if you have ROC on and not a lot of sharing, things actually scale quite nicely. If you don’t have ROC on it wasn’t nearly as good ... At that point there was a lot of concern about our compiler and we could not slow down our compilers. Unfortunately the compilers were exactly the programs that ROC did not do well at. We were seeing 30, 40, 50% and more slowdowns and that was unacceptable. Go is proud of how fast its compiler is."