5 ms·

Golang: sub-millisecond GC pause on production 18gb heap

119 points by eis 10y ago

reacharavindh 10y ago
Just started learning Go. How is that possible? Does this mean the GC tries to do its thing without a pause? Or is it not doing its thing? What is happening here?
- sjwhitworth 10y ago
  https://groups.google.com/forum/?fromgroups#!topic/golang-dev/Ab1sFeoZg_8 https://groups.google.com/forum/?fromgroups#!topic/golang-de...
  - reacharavindh 10y ago
    Thanks for the link. Found the proposal a really good read. Direct link : https://github.com/golang/proposal/blob/master/design/17503-eliminate-rescan.md https://github.com/golang/proposal/blob/master/design/17503-...
karussell 10y ago
Would be interesting to have a raw comparison to the JVM e.g. to the new GC1 which can handle also >10GB with low pauses. I know from experience (with the JVM) that you can get nice pauses on average, but what about the maximum values?
- bascule 10y ago
  Azul's C4 collector can handle 2TB heaps with "pauseless" (<50 microsecond) operation: https://www.azul.com/press_release/azul-systems-extends-support-for-in-memory-computing-architectures-with-new-release-of-zing-runtime-for-java/ https://www.azul.com/press_release/azul-systems-extends-supp...
  - karussell 10y ago
    I know and I really like the achievements! Also there are other JVM GCs usable for RT application. But I would preferable have a comparison between two open source systems ie. go and openjdk8/9 :)
  - brianwawok 10y ago
    At what cost on happy path code? Azul has some other problems with the JIT that make happy path code slower in exchange for better GCs.
- jakewins 10y ago
  I spent a lot of the past six years dreaming nightmares of the unpredictability of CMS and, to a lesser extent, G1. So it's been with extreme skepticism that I've received the Go teams claims of how the Go GC performs. Maximum STW of <1ms? Uhh, ok. The best-of-breed open Java GCs have targets of <100ms on big heaps, 1ms ceiling sounded insane. However, diving into it, the problems solved by the Go GC and those solved by the Java GC are very different in at least two major ways: - Go has substantially less dynamic runtime behavior; there's no JIT, no runtime loading of new code, very limited reflection, &c. - The Go GC does not do heap compaction Both of these things mean: a) Objects do not randomly move about the heap and b) code does not randomly redefine itself. Which, the Go team have made their bed, just like the Java team have, and the Go bedroom looks a heck of a lot more comfy to sleep in. Building a fast GC for Go is a massively simpler problem than building a GC for Java, hence the claims they are making of worst-case scenarios are not completely off the wall. Now, that doesn't mean they should be taken at face value. Notably, the Go GC still has to decide when to start a cycle, and still needs to stop the application if it runs out of heap due to starting a cycle too late. I would suspect that you can construe a test case with a very large heap and a random and extremely varying allocation rate that could cause the Go runtime to explode just fine. Likewise, the time to complete a GC cycle is (at best) linear to the heap size of Go - as the heap grows, the time to reclaim memory does as well, meaning you need to keep larger margins in the heap for floating garbage waiting to be reclaimed.
  - SomeCallMeTim 10y ago
    You've got some great points, but there's one more dimension to why GC in Java hurts performance as much as it does: Java, by the design of its standard libraries and by the conventions that many (most?) Java developers use, just tends to allocate memory for every single operation. I was writing the Java part of a cross-platform game engine once, and I was trying to eliminate all dynamic allocations during the game (front-loading all allocations and then never touching the heap). Did some profiling and determined I'd missed something. The allocation turned out to be in a call to get the current system timer (in milliseconds, if I recall). I was calling a Java function that returned a Java long int, probably straight from an OS call that returned a 64-bit int, and somewhere in that call it was allocating an Object. Ripped out the Java library call, replaced it with a JNI call to a two line C function that returned the result of the corresponding OS call. No more dynamic allocations. It's just part of the philosophy of Java to allocation things everywhere. So it's not only hard to make a GC fast for architectural reasons; the GC also tends to be worked harder by the way Java code tends to be written.
- BenoitP 10y ago
  > Would be interesting to have a raw comparison to the JVM e.g. to the new GC1 And with Shenandoah [1] too! Also, everyone talks about heap sizes; which is kind of the size of the problem if you let it accumulate. I would be much more interested in having these latencies measured in the context of sustained allocation throughput, with varying object lifetime lengths. How much the system can take while staying afloat? Here is the most performant gc: never allow to allocate anything. Go programming constrains heavily towards message passing. I guess this is some kind of tradeoff. I really hope it still makes a lot of new apps possible (40k users concurrently interacting multiplayer games, anyone?). [1] http://openjdk.java.net/jeps/189 http://openjdk.java.net/jeps/189
  - pjmlp 10y ago
    Game studios have been using GC enabled languages for their servers for quite a while now. For example Wooga has quite a few games using Erlang, Hallo used C#/Project Orleans, just to cite two examples.
    - BenoitP 10y ago
      Oh, definitely. And Whatsapp is able to maintain 2.8M connections, > 200k msgs/sec on one machine with Erlang [1]. I meant not in an embarrassingly parallel worload. The 40k would be with interacting with each other in realtime, and not by groups maxed at 10-20. Each of the 40k being impacted by the other 40k's actions; with an all-reduce step between them at each tick. Each of them getting a different overview of the other 40k. Imagine a FPS where users receive updates on others' position 60 times per second if they are close, 1 time per second if they are far away, and none if the player is not looking at them. This could enable vastly larger games. The question at hand would be: Go and Erlang's GC enable great latencies; but can their programming model still enable this worload easily? What kind of degree of shared mutable state do we need? [1] http://highscalability.com/blog/2014/2/26/the-whatsapp-architecture-facebook-bought-for-19-billion.html http://highscalability.com/blog/2014/2/26/the-whatsapp-archi...
- rogerdpack 10y ago
  Here's one comparison with G1: https://blog.pusher.com/golangs-real-time-gc-in-theory-and-practice/ https://blog.pusher.com/golangs-real-time-gc-in-theory-and-p... where G1 still had some reasonably large pauses FWIW. Which is surprising isn't its billing to not do that? But FWIW.
revelation 10y ago
Well, congratulations on the bragging rights?! That is magnitudes too long for actual real-time or low latency use. It is squarely in a no-mans-land of applications where the worst-case is still too much for games or sound but all other uses didn't care very much for GC pause in the first place.
- throwaway2016a 10y ago
  That's great... but 1. A 18 gigabyte heap puts it magnitudes larger than a game, presumably performance is better for smaller apps 2. I write web application servers and I very much care about GC latency
- hamandcheese 10y ago
  For gaming, assuming a target of 60fps, a 1ms GC pause still leaves 15.6ms to render the next frame. Which is 93% of the available time.
  - socmag 10y ago
    Or even better.. I've written 3D/Game Engines engines in languages with GC based runtimes that run at 1500fps with no GC pauses. How to do it? Easy... don't create garbage! So yes, it is doable. PITA but doable Certainly helps if you have a good generational GC. That said, if you can force a compaction once a frame and it's pretty stable at a millisecond, I guess that's more than fine for a lot of stuff.
    - nameless912 10y ago
      Exactly this. In Java, it's definitely possible to almost completely avoid garbage, and the GC is tunable enough that you could consistently have the GC run once per frame, which then makes it just a normal part of the frame timing if you have a general idea of what is being created. Object pooling is magical, yo. I imagine in Golang, this would only get easier as the pauses are super short and I feel more control over the memory allocation (whether this is true or not is an open question).
en4bz 10y ago
GC time is proportional to the number of objects tracked by the GC not the size of the heap, although they do tend to be related. Without knowledge of how many objects are being tracked this is a useless metric.
- socmag 10y ago
  Hehe I thought that as well when I read the headline. "We have three root objects of 6GB a piece!"... ummm :P I'm sure they must be talking about something other than that, but it did make me giggle a bit.
- tapirl 10y ago
  What the the tweet mentioned is GC pause time, not GC time.

119 points by eis 10y ago

Just started learning Go. How is that possible? Does this mean the GC tries to do its thing without a pause? Or is it not doing its thing? What is happening here?

sjwhitworth 10y ago
https://groups.google.com/forum/?fromgroups#!topic/golang-dev/Ab1sFeoZg_8 https://groups.google.com/forum/?fromgroups#!topic/golang-de...
- reacharavindh 10y ago
  Thanks for the link. Found the proposal a really good read. Direct link : https://github.com/golang/proposal/blob/master/design/17503-eliminate-rescan.md https://github.com/golang/proposal/blob/master/design/17503-...

karussell 10y ago

Would be interesting to have a raw comparison to the JVM e.g. to the new GC1 which can handle also >10GB with low pauses. I know from experience (with the JVM) that you can get nice pauses on average, but what about the maximum values?

bascule 10y ago
Azul's C4 collector can handle 2TB heaps with "pauseless" (<50 microsecond) operation: https://www.azul.com/press_release/azul-systems-extends-support-for-in-memory-computing-architectures-with-new-release-of-zing-runtime-for-java/ https://www.azul.com/press_release/azul-systems-extends-supp...
- karussell 10y ago
  I know and I really like the achievements! Also there are other JVM GCs usable for RT application. But I would preferable have a comparison between two open source systems ie. go and openjdk8/9 :)
- brianwawok 10y ago
  At what cost on happy path code? Azul has some other problems with the JIT that make happy path code slower in exchange for better GCs.
jakewins 10y ago
I spent a lot of the past six years dreaming nightmares of the unpredictability of CMS and, to a lesser extent, G1. So it's been with extreme skepticism that I've received the Go teams claims of how the Go GC performs. Maximum STW of <1ms? Uhh, ok. The best-of-breed open Java GCs have targets of <100ms on big heaps, 1ms ceiling sounded insane. However, diving into it, the problems solved by the Go GC and those solved by the Java GC are very different in at least two major ways: - Go has substantially less dynamic runtime behavior; there's no JIT, no runtime loading of new code, very limited reflection, &c. - The Go GC does not do heap compaction Both of these things mean: a) Objects do not randomly move about the heap and b) code does not randomly redefine itself. Which, the Go team have made their bed, just like the Java team have, and the Go bedroom looks a heck of a lot more comfy to sleep in. Building a fast GC for Go is a massively simpler problem than building a GC for Java, hence the claims they are making of worst-case scenarios are not completely off the wall. Now, that doesn't mean they should be taken at face value. Notably, the Go GC still has to decide when to start a cycle, and still needs to stop the application if it runs out of heap due to starting a cycle too late. I would suspect that you can construe a test case with a very large heap and a random and extremely varying allocation rate that could cause the Go runtime to explode just fine. Likewise, the time to complete a GC cycle is (at best) linear to the heap size of Go - as the heap grows, the time to reclaim memory does as well, meaning you need to keep larger margins in the heap for floating garbage waiting to be reclaimed.
- SomeCallMeTim 10y ago
  You've got some great points, but there's one more dimension to why GC in Java hurts performance as much as it does: Java, by the design of its standard libraries and by the conventions that many (most?) Java developers use, just tends to allocate memory for every single operation. I was writing the Java part of a cross-platform game engine once, and I was trying to eliminate all dynamic allocations during the game (front-loading all allocations and then never touching the heap). Did some profiling and determined I'd missed something. The allocation turned out to be in a call to get the current system timer (in milliseconds, if I recall). I was calling a Java function that returned a Java long int, probably straight from an OS call that returned a 64-bit int, and somewhere in that call it was allocating an Object. Ripped out the Java library call, replaced it with a JNI call to a two line C function that returned the result of the corresponding OS call. No more dynamic allocations. It's just part of the philosophy of Java to allocation things everywhere. So it's not only hard to make a GC fast for architectural reasons; the GC also tends to be worked harder by the way Java code tends to be written.
BenoitP 10y ago
> Would be interesting to have a raw comparison to the JVM e.g. to the new GC1 And with Shenandoah [1] too! Also, everyone talks about heap sizes; which is kind of the size of the problem if you let it accumulate. I would be much more interested in having these latencies measured in the context of sustained allocation throughput, with varying object lifetime lengths. How much the system can take while staying afloat? Here is the most performant gc: never allow to allocate anything. Go programming constrains heavily towards message passing. I guess this is some kind of tradeoff. I really hope it still makes a lot of new apps possible (40k users concurrently interacting multiplayer games, anyone?). [1] http://openjdk.java.net/jeps/189 http://openjdk.java.net/jeps/189
- pjmlp 10y ago
  Game studios have been using GC enabled languages for their servers for quite a while now. For example Wooga has quite a few games using Erlang, Hallo used C#/Project Orleans, just to cite two examples.
  - BenoitP 10y ago
    Oh, definitely. And Whatsapp is able to maintain 2.8M connections, > 200k msgs/sec on one machine with Erlang [1]. I meant not in an embarrassingly parallel worload. The 40k would be with interacting with each other in realtime, and not by groups maxed at 10-20. Each of the 40k being impacted by the other 40k's actions; with an all-reduce step between them at each tick. Each of them getting a different overview of the other 40k. Imagine a FPS where users receive updates on others' position 60 times per second if they are close, 1 time per second if they are far away, and none if the player is not looking at them. This could enable vastly larger games. The question at hand would be: Go and Erlang's GC enable great latencies; but can their programming model still enable this worload easily? What kind of degree of shared mutable state do we need? [1] http://highscalability.com/blog/2014/2/26/the-whatsapp-architecture-facebook-bought-for-19-billion.html http://highscalability.com/blog/2014/2/26/the-whatsapp-archi...
rogerdpack 10y ago
Here's one comparison with G1: https://blog.pusher.com/golangs-real-time-gc-in-theory-and-practice/ https://blog.pusher.com/golangs-real-time-gc-in-theory-and-p... where G1 still had some reasonably large pauses FWIW. Which is surprising isn't its billing to not do that? But FWIW.

revelation 10y ago

Well, congratulations on the bragging rights?! That is magnitudes too long for actual real-time or low latency use. It is squarely in a no-mans-land of applications where the worst-case is still too much for games or sound but all other uses didn't care very much for GC pause in the first place.

throwaway2016a 10y ago
That's great... but 1. A 18 gigabyte heap puts it magnitudes larger than a game, presumably performance is better for smaller apps 2. I write web application servers and I very much care about GC latency
hamandcheese 10y ago
For gaming, assuming a target of 60fps, a 1ms GC pause still leaves 15.6ms to render the next frame. Which is 93% of the available time.
- socmag 10y ago
  Or even better.. I've written 3D/Game Engines engines in languages with GC based runtimes that run at 1500fps with no GC pauses. How to do it? Easy... don't create garbage! So yes, it is doable. PITA but doable Certainly helps if you have a good generational GC. That said, if you can force a compaction once a frame and it's pretty stable at a millisecond, I guess that's more than fine for a lot of stuff.
  - nameless912 10y ago
    Exactly this. In Java, it's definitely possible to almost completely avoid garbage, and the GC is tunable enough that you could consistently have the GC run once per frame, which then makes it just a normal part of the frame timing if you have a general idea of what is being created. Object pooling is magical, yo. I imagine in Golang, this would only get easier as the pauses are super short and I feel more control over the memory allocation (whether this is true or not is an open question).

en4bz 10y ago

GC time is proportional to the number of objects tracked by the GC not the size of the heap, although they do tend to be related. Without knowledge of how many objects are being tracked this is a useless metric.

socmag 10y ago
Hehe I thought that as well when I read the headline. "We have three root objects of 6GB a piece!"... ummm :P I'm sure they must be talking about something other than that, but it did make me giggle a bit.
tapirl 10y ago
What the the tweet mentioned is GC pause time, not GC time.