7 ms·

That statement has been said for pretty much every release of any GC. I'd say the opposite is true. Relying on GC requires significant extra work. Because you

by tjoff 6y ago

That statement has been said for pretty much every release of any GC. I'd say the opposite is true. Relying on GC requires significant extra work. Because you always need to think about memory (exception: small script-like applications). The only thing a GC does is that it enable you to not think about it, but the moment you don't you will write bad code and realize it was a disservice all along. And by then it is too late. So in a GC language you need to constantly be aware of when you take something for granted. Which is more work than just doing it manually yourself.

RhodesianHunter 6y ago
This is not really true at all though, nor is it how people who design performance intensive systems go about it. generally you design your application the way you normally would, do some testing, and then go back through the hot path and make some adjustments.
pron 6y ago
First, a GC allows you to abstract over memory allocations (i.e. hide them). This makes maintaining code over time significantly easier, as allocation becomes a hidden implementation detail. Second, while there have been similar claims made in the past, they always apply for a certain target throughput and latency. GCs are constantly making great strides in that regard. Next year, ZGC will have a <1ms worst-case latency for pretty serious allocation rates, and with an acceptable hit to throughput. As you can see in this post and previous ones, G1 offers great throughput with acceptable latencies.
- jakewins 6y ago
  This GC you speak of that hides allocations as an implementation detail sounds excellent! Every GC I've worked with - and my day job is orchstrating several thousand JVMs - simply moves the time when you need to think of allocations somewhere else. Either it moves it into PagerDuty, like G1, or it moves it into your GCP bill like Shenandoa and ZGC. The trade-off has always been latency-throughput-footprint, nothing I've seen yet has changed that. The innovation in Rust is realizing you can do all the tracing work at compile time.
  - pron 6y ago
    An implementation detail doesn't mean that you don't think about it. We think about implementations all the times. It means that it's a detail, and changing a subroutine's allocation pattern does not create a ripple-effect throughout the code in its transitive consumers. > The innovation in Rust is realizing you can do all the tracing work at compile time. This is spectacularly false. For all non-trivial allocation/deallocation patterns, Rust also uses a runtime reference-counting GC, which is significantly slower than the tracing GCs you find in OpenJDK. The benefit comes from not relying on it too much, but this comes at a considerable cost of lost abstraction, which means more costly maintenance over the years. This is the same for all low-level languages; the difference Rust brings is that it (conservatively!) checks for memory access errors. Another difference is that most people who talk about Rust haven't actually written a significant application in it and had to maintain it for years. I'm not saying it's impossible -- people do this for C and C++, which make a similar tradeoff in this regard -- but it does come at a substantial cost.
    - jakewins 6y ago
      At this point I think we are arguing somewhat subjective things. I used to be a hard-core zealot for the JVM as the performance platform of the future - and did talks arguing just as you are here why HotSpot outperforms $LANGUAGE in real applications. I feel like I'm hearing myself in your argument.. I wrote a signficant portion of the Neo4j storage engine, which is in Java. Now I'm writing another database engine in Rust (sidenote: not for replacing the Neo4j engine, just because it's interesting). Arguably database engines qualify as "significant applications". I find - subjectively: - Maintaining performant code in Rust is easier. I do the same patterns as I did in Java, except it doesn't rely on easy-to-break assumptions of how HotSpot happens to work (ex: stack allocations) - Like you said elsewhere, the issue is often about fragmentation, ultimately stemming from object churn. I find that Rust makes it, culturally perhaps, easier to maintain low allocation code than Java. (sidenote: I think these two points is also why Go code generally has "better GC behavior"; Go doesn't have a better GC, it has a language that encourages less heap allocation) - The engine I'm writing in Rust is faster than anything I've written in Java and - critically - runs for days and months without notable stalls. Come to the dark side!
- haxen 6y ago
  > ZGC will have a <1ms worst-case latency In our benchmarks we never saw a GC pause of more than 2 ms on either ZGC or Shenandoah, but the end-to-end latency, the one the user cares about, is impacted by much more than a single GC pause. Sometimes there would be several pauses in a rapid sequence, or just the background GC thread would do too much work at once. Even after dedicating a core or two to the GC, you still face the issues of cache pollution and RAM throughput stealing that heap walking incurs.
  - pron 6y ago
    Yep, that's because concurrent stack processing (https://openjdk.java.net/jeps/376 https://openjdk.java.net/jeps/376) hasn't been merged yet, nor has the changes to the VM's internal references. Both of those are coming within a year. Until then, there's still significant work done by the GC at safepoints.
    - haxen 6y ago
      A month ago we benchmarked against the master of the ZGC repo. There wasn't a big difference.
      - pron 6y ago
        Sorry, I misread your previous comment. The planned improvements that would make the biggest difference are making ZGC generational, and improving scalar-replacement (or doing some sort of stack allocation).
- CyberDildonics 6y ago
  > This makes maintaining code over time significantly easier, as allocation becomes a hidden implementation detail. You say that, but there seem to be no end to the stories of people spending enormous amount of time fighting the gc. It is not difficult to avoid heap allocations in other languages as well as freeing them deterministically.
  - pron 6y ago
    First, you say it as if there aren't event bigger struggles without a GC. So the comparison is not "I'm fighting with the GC vs. I'm not" but "I'm fighting with the GC vs fighting other things, which is better?" So you hear those stories because those are the stories people who use a GC can tell. Second, as this blog post series shows, the "fight" is not what it used to be. You don't really need to control allocation any more until your rates are really high. Java's GCs have just gotten so much better in JDK 14 and beyond.
    - CyberDildonics 6y ago
      > First, you say it as if there aren't event bigger struggles without a GC. I'm extremely skeptical about that. My experience is that with modern C++ you lose very little elegance and gain a huge amount of control by giving up a garbage collector. Memory management becomes a very minor problem. The vast majority of memory allocations are avoided and those that need to be there can be done ahead of time. I have never heard anyone writing a latency sensitive program in C++ (games, trading etc.) say that their life would be easier if they were using a gc or that they wished they could do it in java. I have however seen decades of people talking about all the extreme lengths and rabbit holes they go down to deal with the java gc. From a broader perspective, pretty much any language with a gc ends up having a perpetual conversation around how the next gc will solve the problems with the current gc. You can see it in java, go, julia, and D. The only one I never hear about is LuaJIT, but maybe I just haven't seen it or maybe the expectations are lower.
      - imtringued 6y ago
        Lua without a JIT is just a glue language. The expectations are low and LuaJIT is easily smashing them because a lot of Lua code is/was written for the original interpreter.
dtech 6y ago
This is only the case when latency is important below some threshold†. There is a very large class of applications that can work with that limitation, and in that case a GC can free up programmer mind-space by letting him mostly forget about memory management. You're optimizing for productivity of the developer instead of hardware usage. Rust or other upcoming or future languages might change that. † The threshold varies between VM and GC, but usually <10ms is easily achievable
- pron 6y ago
  ZGC's pause latency is at a point where it rivals, or about to rival, nondeterministic pauses by the OS. So unless you're running on a realtime kernel, GC is not an issue any more as far as latency is concerned. The only real, serious cost for modern GCs is RAM overhead.
  - imtringued 6y ago
    I'm not convinced to be honest. Even with tiny heaps like 32MB JVM processes with practically nothing but a hello world micronaut project consume up to 200MB of RAM. There is more to this than just the dumb GC which always predictably hits the configured maximum or if there is no maximum it will just gobble up your most of your system. I once saw an instance of tomcat allocate 30GB to itself on a staging system with at most one user at a time simply because the machine had 128GB RAM. 30GB for nothing. I personally dislike the JVM. However I have no problem with Java or any JVM derived languages but it simply doesn't matter how much you hyper optimize your code if the underlying VM is junk.
    - pron 6y ago
      The way RAM is managed in Java and in C is very different; neither is better than the other, but it's different, and you don't seem to understand the Java model. In that model, you give an application a maximum heap, which the program will use regardless of the minimum it needs to improve its performance. You want it to use less RAM? Just tell it to. Having said that, very recently Java has started to return RAM to the OS if it's just sitting idle. See https://malloc.se/blog/zgc-softmaxheapsize https://malloc.se/blog/zgc-softmaxheapsize

by tjoff 6y ago

RhodesianHunter 6y ago

This is not really true at all though, nor is it how people who design performance intensive systems go about it. generally you design your application the way you normally would, do some testing, and then go back through the hot path and make some adjustments.

pron 6y ago

First, a GC allows you to abstract over memory allocations (i.e. hide them). This makes maintaining code over time significantly easier, as allocation becomes a hidden implementation detail. Second, while there have been similar claims made in the past, they always apply for a certain target throughput and latency. GCs are constantly making great strides in that regard. Next year, ZGC will have a <1ms worst-case latency for pretty serious allocation rates, and with an acceptable hit to throughput. As you can see in this post and previous ones, G1 offers great throughput with acceptable latencies.

jakewins 6y ago
This GC you speak of that hides allocations as an implementation detail sounds excellent! Every GC I've worked with - and my day job is orchstrating several thousand JVMs - simply moves the time when you need to think of allocations somewhere else. Either it moves it into PagerDuty, like G1, or it moves it into your GCP bill like Shenandoa and ZGC. The trade-off has always been latency-throughput-footprint, nothing I've seen yet has changed that. The innovation in Rust is realizing you can do all the tracing work at compile time.
- pron 6y ago
  An implementation detail doesn't mean that you don't think about it. We think about implementations all the times. It means that it's a detail, and changing a subroutine's allocation pattern does not create a ripple-effect throughout the code in its transitive consumers. > The innovation in Rust is realizing you can do all the tracing work at compile time. This is spectacularly false. For all non-trivial allocation/deallocation patterns, Rust also uses a runtime reference-counting GC, which is significantly slower than the tracing GCs you find in OpenJDK. The benefit comes from not relying on it too much, but this comes at a considerable cost of lost abstraction, which means more costly maintenance over the years. This is the same for all low-level languages; the difference Rust brings is that it (conservatively!) checks for memory access errors. Another difference is that most people who talk about Rust haven't actually written a significant application in it and had to maintain it for years. I'm not saying it's impossible -- people do this for C and C++, which make a similar tradeoff in this regard -- but it does come at a substantial cost.
  - jakewins 6y ago
    At this point I think we are arguing somewhat subjective things. I used to be a hard-core zealot for the JVM as the performance platform of the future - and did talks arguing just as you are here why HotSpot outperforms $LANGUAGE in real applications. I feel like I'm hearing myself in your argument.. I wrote a signficant portion of the Neo4j storage engine, which is in Java. Now I'm writing another database engine in Rust (sidenote: not for replacing the Neo4j engine, just because it's interesting). Arguably database engines qualify as "significant applications". I find - subjectively: - Maintaining performant code in Rust is easier. I do the same patterns as I did in Java, except it doesn't rely on easy-to-break assumptions of how HotSpot happens to work (ex: stack allocations) - Like you said elsewhere, the issue is often about fragmentation, ultimately stemming from object churn. I find that Rust makes it, culturally perhaps, easier to maintain low allocation code than Java. (sidenote: I think these two points is also why Go code generally has "better GC behavior"; Go doesn't have a better GC, it has a language that encourages less heap allocation) - The engine I'm writing in Rust is faster than anything I've written in Java and - critically - runs for days and months without notable stalls. Come to the dark side!
haxen 6y ago
> ZGC will have a <1ms worst-case latency In our benchmarks we never saw a GC pause of more than 2 ms on either ZGC or Shenandoah, but the end-to-end latency, the one the user cares about, is impacted by much more than a single GC pause. Sometimes there would be several pauses in a rapid sequence, or just the background GC thread would do too much work at once. Even after dedicating a core or two to the GC, you still face the issues of cache pollution and RAM throughput stealing that heap walking incurs.
- pron 6y ago
  Yep, that's because concurrent stack processing (https://openjdk.java.net/jeps/376 https://openjdk.java.net/jeps/376) hasn't been merged yet, nor has the changes to the VM's internal references. Both of those are coming within a year. Until then, there's still significant work done by the GC at safepoints.
  - haxen 6y ago
    A month ago we benchmarked against the master of the ZGC repo. There wasn't a big difference.
    - pron 6y ago
      Sorry, I misread your previous comment. The planned improvements that would make the biggest difference are making ZGC generational, and improving scalar-replacement (or doing some sort of stack allocation).
CyberDildonics 6y ago
> This makes maintaining code over time significantly easier, as allocation becomes a hidden implementation detail. You say that, but there seem to be no end to the stories of people spending enormous amount of time fighting the gc. It is not difficult to avoid heap allocations in other languages as well as freeing them deterministically.
- pron 6y ago
  First, you say it as if there aren't event bigger struggles without a GC. So the comparison is not "I'm fighting with the GC vs. I'm not" but "I'm fighting with the GC vs fighting other things, which is better?" So you hear those stories because those are the stories people who use a GC can tell. Second, as this blog post series shows, the "fight" is not what it used to be. You don't really need to control allocation any more until your rates are really high. Java's GCs have just gotten so much better in JDK 14 and beyond.
  - CyberDildonics 6y ago
    > First, you say it as if there aren't event bigger struggles without a GC. I'm extremely skeptical about that. My experience is that with modern C++ you lose very little elegance and gain a huge amount of control by giving up a garbage collector. Memory management becomes a very minor problem. The vast majority of memory allocations are avoided and those that need to be there can be done ahead of time. I have never heard anyone writing a latency sensitive program in C++ (games, trading etc.) say that their life would be easier if they were using a gc or that they wished they could do it in java. I have however seen decades of people talking about all the extreme lengths and rabbit holes they go down to deal with the java gc. From a broader perspective, pretty much any language with a gc ends up having a perpetual conversation around how the next gc will solve the problems with the current gc. You can see it in java, go, julia, and D. The only one I never hear about is LuaJIT, but maybe I just haven't seen it or maybe the expectations are lower.
    - imtringued 6y ago
      Lua without a JIT is just a glue language. The expectations are low and LuaJIT is easily smashing them because a lot of Lua code is/was written for the original interpreter.

dtech 6y ago

This is only the case when latency is important below some threshold†. There is a very large class of applications that can work with that limitation, and in that case a GC can free up programmer mind-space by letting him mostly forget about memory management. You're optimizing for productivity of the developer instead of hardware usage. Rust or other upcoming or future languages might change that. † The threshold varies between VM and GC, but usually <10ms is easily achievable

pron 6y ago
ZGC's pause latency is at a point where it rivals, or about to rival, nondeterministic pauses by the OS. So unless you're running on a realtime kernel, GC is not an issue any more as far as latency is concerned. The only real, serious cost for modern GCs is RAM overhead.
- imtringued 6y ago
  I'm not convinced to be honest. Even with tiny heaps like 32MB JVM processes with practically nothing but a hello world micronaut project consume up to 200MB of RAM. There is more to this than just the dumb GC which always predictably hits the configured maximum or if there is no maximum it will just gobble up your most of your system. I once saw an instance of tomcat allocate 30GB to itself on a staging system with at most one user at a time simply because the machine had 128GB RAM. 30GB for nothing. I personally dislike the JVM. However I have no problem with Java or any JVM derived languages but it simply doesn't matter how much you hyper optimize your code if the underlying VM is junk.
  - pron 6y ago
    The way RAM is managed in Java and in C is very different; neither is better than the other, but it's different, and you don't seem to understand the Java model. In that model, you give an application a maximum heap, which the program will use regardless of the minimum it needs to improve its performance. You want it to use less RAM? Just tell it to. Having said that, very recently Java has started to return RAM to the OS if it's just sitting idle. See https://malloc.se/blog/zgc-softmaxheapsize https://malloc.se/blog/zgc-softmaxheapsize