17 ms·

Go, don't collect my garbage

272 points by jgrahamc 9y ago

littlestymaar 9y ago
This is a good illustration that Go isn't as simple as its proponents like to praise[1]. When you face complex engineering challenges, you need tools with a certain amount of complexity. [1] especially when comparing to Java when it comes to GC customization.
- mining 9y ago
  I don't know. That seemed like a fairly simple exploration of the problem and an easy library function to solve it. Would it be much simpler in any other language?
  - dboreham 9y ago
    Simpler would be "just work like I expect".
    - zaarn 9y ago
      Doesn't work with GCs. As another comment in this thread explained, GCs have CPU and Memory tradeoffs and you can easily make a GC that would work excellent in the OP benchmark but would suffer severely under other workloads.
    - merlish 9y ago
      I think the expectation the Go lot have been working towards is the expectation that individual pauses are very short. Contrast to a Java server GC, aiming towards overall efficiency. (Tell me if I'm wrong, I'm no expert.)
      - weberc2 9y ago
        Yes, Go optimizes for low latency and Java for high throughput.
        pjmlp 9y ago
        Depends on the Java GC being used.
        weberc2 9y ago
        Granted, but the stock GC is optimized for throughput.
        pjmlp 9y ago
        The stock GC on OpenJDK, there are other JVMs to choose from.
    - falcolas 9y ago
      What about the use cases where what you expect is not what I expect? GC is, as a sibling comment says, about compromises around memory safety.
    - weberc2 9y ago
      As another commented posted elsewhere, GC is all about tradeoffs, and for a govern set of GC tradeoffs there will always be pathological cases. Another GC might work out of the box for this workload, but perform very badly for a workload at which Go's GC excels. As others have already mentioned, the fact that he was so easily able to tune the GC is a testament to Go's simplicity. There are lots of good criticisms of Go to be had, but it's runtime is pretty remarkable, in my opinion.
  - lmm 9y ago
    It's a simple problem with a simple fix, from a very simple benchmark - but it's already reaching the limits of how much tuning Go offers. If even such a basic benchmark needed one GC tuning knob, that seems suggestive that maybe a larger and more complex program really would need all the knobs that Java offers, the ones that Go claims are unnecessary. Or maybe not - maybe that one knob really can solve all the problems. But I really was surprised that anyone would ever need to actively configure a GC for such a simple use case, so I've learnt something at least.
    - weberc2 9y ago
      I think most pathological cases can be reduced to similarly simple examples. I'm sure these exist for other GCs as well.
      - stmw 9y ago
        Yes, the take-away here is not that Go is particularly bad, but that GCs are problematic. Notably, GC != automatic memory management.
        littlestymaar 9y ago
        > Notably, GC != automatic memory management. Actually, GC == automatic memory management, just not automatic performant memory management
        weberc2 9y ago
        That's not a reasonable conclusion to draw. You would need to preallocate to maximize performance regardless of GC in this case, and in other cases, the performance and determinism gaps between GC languages and non-GC languages is almost entirely due to tertiary factors. For example, GC languages are more likely to prioritize friendliness over raw performance in other areas--language features, compiler optimizations, standard library design, etc. Non-GC languages really only need to appeal to the high-performance and high-determinism crowds.
    - stmw 9y ago
- lucozade 9y ago
  Funny what different people get from these things. I was most impressed that he managed to get very close to perfect scalability by twiddling the one knob on offer. Java GC customisation is an art. This analysis was entirely mechanical. If the author wasn't just lucky i.e. that this is sufficient for a large number of workload profiles, then that's very good news.
  - marcosdumay 9y ago
    The problem of software knobs is that the one that helps you is almost never there, and you usually can't get similar functionality from mixing the ones that are there. That said, there are exceptions. I don't think GC is something simple enough to be an exception, but those exist.
  - tveita 9y ago
    The main problem is that it isn't a very good knob for this use case, as "ratio of freshly allocated data to live data remaining" is only incidental to the goal of the person writing the program. Therefore you'd expect to have to retune it due to unrelated changes, like if you allocated a big chunk of static memory to hold results. I expect Go's current behaviour is right for memory-intensive programs, where "limit the wasted memory as a ratio of the actual working set" is a reasonable high-level goal. For a CPU intensive program you might have preferred a "max x% CPU overhead" knob, but just a minimum memory size would work, as this is mostly an artifact of the very small memory usage of benchmark program. Java's G1 has "max pause delay" as their preferred knob, but Go has stuck that knob on low since around version 1.6, which is a good choice for what is essentially a language for network services.
- merlish 9y ago
  Interesting! IMO it showed that garbage collection isn't some silver bullet (of course), but that with a kick to a quite coarse control it's possible to achieve very good performance indeed. I am surprised that the difference is so great - and that the GC seems to think it's a good idea to keep collecting 4 MB of data at a time - but I guess the Go team have been tuning heavily for interactive performance i.e. small pauses. I guess in a situation like this you'd prefer something more like 'server' GC - maybe wait until you've allocated 1 GB then run through and collect everything you can. In games, I wonder if you'd like to turn the GC off during frame rendering then collect while presenting the frame/waiting for VSync. For soft real-time audio - synthesizing a few samples at a time - what would you do? OpenAL wants you to queue up thousands or tens of thousands of samples, but that sucks if you need to react very quickly. If anyone has any ideas, please say. I'd love to do soft real-time audio in a thread even in e.g. C#. (Edit: Maybe JIT some code that doesn't allocate? Or even have a separate interpreter written in C that has a constant-ish upper performance bound & no malloc.)
- Thaxll 9y ago
  Having worked with Java GC it's just a nightmare to tune. I prefer languages that have almost no options for GC tuning.
  - rrampage 9y ago
    On Java 8+, you can use G1 GC which has very few knobs. On the other hand, CMS GC has a lot of knobs you can tune depending on your workload. JVM (Hotspot in this case) believes in flexibility over a one-size fits all. YMMV.
    - stmw 9y ago
      In other words, there are actually knobs that have you choose the GC, and then choose settings within the GC.
  - erik_seaberg 9y ago
    Tuning is a hassle, but when the alternative is rewriting my important code over and over until I hit the right tradeoff between fast and readable, I'll take it.
jasode 9y ago
Whenever I see GC, I immediately think of tradeoffs. One can optimize a GC algorithm (or GC configuration setting) to work well for one type of workload. Likewise, one can always create a pathological workload for any GC configuration because GC has unavoidable tradeoff between cpu time and memory footprint. Therefore, I read his article with the intent of placing his findings in the "taxonomy" of GC tradeoffs. In that vein, I noticed that his benchmarks and graphs do not include measurements for memory footprint. Instead, it has throughput statistics such as "ops/sec" or "sign/second". I think I see why working memory size isn't emphasized. He's testing a crypto algorithm "ECDSA-P256 Sign". I'm not familiar with it but I assume we can think of it as a typical hash function[1] that doesn't require unpredictable dynamic allocation of memory as the algorithm processes the bytes of input. (A typical hash function uses the same fixed amount of memory whether hashing 1 kilobyte or 1 terabyte.) If his synthetic benchmark to play with GOGC was using a different type of workload... such as a text parser that stressed the GC with thousands of different sized memory objects, we'd see memory footprint as part of the analysis. [1] https://en.wikipedia.org/wiki/Elliptic_Curve_Digital_Signature_Algorithm https://en.wikipedia.org/wiki/Elliptic_Curve_Digital_Signatu...
- dullgiulio 9y ago
  It's not just a fixed-length hash, but it does computations on the curve points, which are represented by big.Int's, that take a variable size and are likely to end up on the heap. Also, Go crypto makes a heavy use of interface that also make stuff escape to the heap. Of such things are a bottleneck, it might be worthwhile to "un-interface" them.
dboreham 9y ago
This doesn't read like a "GC is slow" problem to me, more like a "GC doesn't cope well in the face of a highly concurrent allocation workload". That's pretty bad though. As soon as you go down the rabbit hole of tweaking knobs to cajole GC into doing what you expect, darkness lies beyond..
- jchw 9y ago
  It sounds like a bug in the GC for it to run so often, but even so, Go's GC has only one knob and they're adament about keeping it that way. There's no rabbit hole to fall into here.
  - Twirrim 9y ago
    There can be advantages (and ZOMG so much confusion) in allowing multiple knobs though. Java takes it to an extreme and we're somewhat in the territory of needing a black box opitimiser, but you can adjust the GC to suit a really wide variety of workloads and requirements. By leaving it to one knob, Google/Go is heavily limiting the uses of the language.
- jerf 9y ago
  All GC algorithms have pessimal cases; I believe there is a mathematical proof of that, though I can never find it on the net through the chatter of all the other stuff about garbage collection. This is a pessimal case for the current Go GC, but that alone doesn't prove much because all algorithms have pessimal cases. (Even "manual memory management" can end up with pessimal cases for the automatic part, which is why you might see an improvement with your "manually managed" program by swapping malloc for jemalloc or something. What people call "manual memory management" still generally includes a lot of automatic stuff.) I believe Go does something like set a threshold of memory use. Once that memory usage is hit, a collection is run. After that collection is run, it uses the new accurate amount of memory used to set the next threshold. The threshold of course start small and have an exponential increase of some sort. When the program first starts, the threshold is set fairly low. If you immediately start allocating lots of things and keeping them, a common case, it grows to a sensible size in a reasonable number of allocations and sweeps (which as you are growing into your initial footprint, amortize out pretty cheap), and in a lot of cases you won't think much about the GC until you are really pushing performance. If, however, you write a program that isn't holding on to much RAM in the steady state, such as a crypto algorithm (which for all their algorithmic complexity typically don't take much RAM to run, as compared to, say, loading an image, and I don't know for sure but it's possible this ends up all stack-allocated), and that algorithm allocates a lot of short-term stuff, perhaps even just allocates for its final answer, then you get into a situation where you are quickly generating enough garbage to cross the initial low threshold, but the GC immediately cleans it all up when it runs, so when it is deciding on what the new threshold should be it still picks a low number, causing it to repeat the cycle indefinitely. Fixes will immediately leap to your mind, but remember you must evaluate your fixes holistically, not as if this is the only case in the world. If it were the only case in the world, it would be an easy problem, and Go never would have exhibited the problem in the first place. In particular, as you consider your "fix", consider also that a real program that used real crypto would be much less likely to see this pessimal case occur, because it is very likely the rest of the program would push the memory thresholds up to something sensible, so the fix of "do nothing" is actually quite likely to work well on non-artifical-benchmark programs. It is likely that the most obvious fix that comes to mind will result in making a program that is deliberately running some small server process or something appearing to leak memory after running for a long period of time, for instance. (I suspect quite strongly in this case that the correct "fix" is to do nothing.)
alkonaut 9y ago
If you make something that is performance critical, and in particular if it's concurrent, the normal rule is you avoid allocating on the heap at all. Use object preallocated data (just operate on massive lists), or use pools of data etc. Any language/runtime will support just allocating a large blob of data and then playing with it with code that looks like C. Effectively once you want to write perf sensitive code in a garbage collectied language you have to stop writing idiomatic code for that language. That can mean using data orientation (SoA for example is a very non-idiomatic thing to do in most C-like languages and especially OO ones). Not using heap allocation at all for any length of time is definitely non-idiomatic in Java/C#/Python/js etc., but that's what you need to do if you want any kind of performance. There are two truths here: 1) Any language can be as fast as C and 2) When they are as fast as C, they also look like C regardless what it was to begin with.
- pjmlp 9y ago
  > Effectively once you want to write perf sensitive code in a garbage collectied language you have to stop writing idiomatic code for that language. Only on GC languages that don't offer proper language features for value types. Sadly most of them failed to gain market share, like Modula-3. Eiffel and D are some of few around that offer such capabilities, and C# has been getting some love from System C# features, with spans, new ref types, and GC regions.
  - alkonaut 9y ago
    Yes, C# is slowly getting there (I'm a desktop C# dev since 14 years so I have waited for these features to creep in). Value types certainly help, but in idiomatic C# if you foreach over something you (sometimes) allocate an iterator on the heap. And it's not clear from the syntax when you do. You just have to know that a List<T> doesn't heap allocate its enumerator but most other types do, for example. These details are mostly hidden from you. Using the low level features in C# (in the past structs, preferably in an AoS layout, then Span<T> etc tomorrow) will effectively make it a "Safe C". When you use those, you can't use idiomatic C (linq, heap allocs etc). You effectively agree to use a completely different subset of the language for your high perf tasks, and when you do, it doesn't look like the C# you have in the rest of your app.
jnordwick 9y ago
As anybody who has done high-performance, low-latency Java and had to deal with writing GC-less Java code, this is no surprise at all. In the Java world we at least had ways around it with things like sun.misc.unsafe, avoiding language features that allocate, pooling, etc. While some of those things can be done in Go, I don't know if all the techniques are possible. Without them, I'm not sure Go can really compete in the arena.
- stmw 9y ago
  Exactly right re: Java. You could just use Rust, and get automatic memory management without GC.
  - abiox 9y ago
    One does not simply "just use Rust".
    - kibwen 9y ago
      Here, allow me to help: https://www.rust-lang.org/en-US/install.html https://www.rust-lang.org/en-US/install.html
dm319 9y ago
Can anyone explain to me why he got better performance with GC turned on (and optimised) rather than off?
- tom_mellior 9y ago
  Judicious garbage collection reuses memory you have already allocated, and for which there may already be entries in caches, page tables, garbage collector metadata, and whatnot. If you never garbage collect, the runtime has to repeatedly allocate new memory from the operating system, and all this additional metadata has to be juggled. If allocating new memory is a system call, that has costs. If you get lots of page faults because you keep using new pages, that has costs as well.
  - emeraldd 9y ago
    It sounds like instead of turning off GC, the algorithm was changed to an "automatic" allocation/deallocation scheme that's closer to what you might see with manual memory management in a C like world. I wonder if the Go runtime has any optimizations to turn things into stack/static allocations in that scenario.
    - tom_mellior 9y ago
      I'm not a Go expert, but my understanding of the blog was that the algorithm or any other aspect of the code was not changed. Only the garbage collection frequency.
      - emeraldd 9y ago
        My comment was in reply to the parent's: > If you never garbage collect, the runtime has to repeatedly allocate new memory from the operating system ...
        tom_mellior 9y ago
        Yeah, that was me. I still don't understand what you're saying. Who is changing what algorithm in what component? If you tell the Go runtime not to garbage collect but keep allocating memory, that memory has to come from somewhere. That somewhere is the operating system.
TazeTSchnitzel 9y ago
The Go team were very proud of their garbage collector that “virtually eliminates” (IIRC) GC pauses. I guess it does that at the cost of severe program slowdown in cases like this? It's all about trade-offs, and I don't trust people who think there's only one thing to optimise for.
- kibwen 9y ago
  While the Go GC is definitely no slouch when it comes to engineering, the messaging around it has not been entirely forthright: https://blog.plan99.net/modern-garbage-collection-911ef4f8bd8e https://blog.plan99.net/modern-garbage-collection-911ef4f8bd...
  - geodel 9y ago
    This blog is more of a rant. Official blog announcements and marketing material are not the place to delve deeply into technical tradeoffs of designing GC. He compares it with Java which is rather ironic considering I have not seen Java official announcements talking about things not done. It would like saying 'We have not been able to deliver value types, module story is still half done, and we couldn't write an official http cient in whole development cycle in Java 9' in release announcement of Java 9. Also there is no data provided to show how bad is Go's GC compare to Java
tuyguntn 9y ago
If any Cloudflare engineers are here, why is Rust not chosen as main language for such use cases?
- acdha 9y ago
  I like Rust too but it's a newer language and the Cloudflare team has a history of using Go since it's targeted pretty squarely at their kind of work. In the case of doing benchmarks, it's especially unhelpful to make that suggestion since the point of a benchmark is to learn where the limits are rather than an invitation to port everything to a different language to avoid a problem which might remain theoretical forever.
  - staticassertion 9y ago
    I don't think they're trying to suggest using rust, just curious why Cloudflare chose a language with a GC for workloads that apparently do not play well with GC. "We already used Go" is a totally valid reason. "It's rare (or as you say potentially never an issue) that we actually need to avoid GC" is another.
    - khc 9y ago
      the context of the benchmark is https://blog.cloudflare.com/arm-takes-wing/ https://blog.cloudflare.com/arm-takes-wing/ (Cloudflare engineer but I have nothing to do with the recent blog posts)
    - acdha 9y ago
      “why is Rust not chosen as main language” seemed like an unnecessarily accusatory way to express that idea, especially given that language choices are usually driven by a number of factors rather than a single benchmark. I've seen a lot of advocacy over the years like that which has seemed to inspire more backlash than adoption. I generally prefer something constructive in the tradition of showing a better way to do something — i.e. in this case, perhaps show a Rust crypto library which delivers better performance so people could judge whether that kind of thing is enough of a draw to be worth migrating or, in the case of Rust, taking advantage of its embedding characteristics.
kbart 9y ago
This is why we will still have C for times to come.
- tebruno99 9y ago
  If you can't figure out how to tune 1 documented option in Go to fix this issue caused by this one specific use case (which he did figure it out and it was resolved).. Then you're probably not using C either.
  - kbart 9y ago
    "Then you're probably not using C either." In fact, I do - daily. I'm not stating that this exact issue is the key reason, but because C is very simple with not much of an abstraction layer between software and hardware, where most such (unexpected) performance issues arises. Yes, it might be a single flag to makes things run normal, but it might also take 1 week in production to find such "magic" bugs.
stmw 9y ago
This kind of tuning ends up being required for all languages that use garbage collection. The more you use it in production, the more you're doing meta programming with environment variables, command-line options and your allocation profiles. The problem is not so much some performance reduction, it's that the performance reduction is often variable and unexpected, as it is in this blog post.
readittwice 9y ago
I would love to see an in-depth analysis why Go's GC behaves this way for this specific benchmark in the first place. Just playing around with different values for `GOGC` seems like guess-work and could just paper over the underlying real issue. I know this probably requires to read the GC's source code, but that would've been certainly been very educational.
Arzh 9y ago
Tuning is part of the job if you need high performance. It can take many forms from code optimization, build flags, platform configurations, to even hardware tweeks. Most people hate this kinda stuff but some of us think it's fun.
mcguire 9y ago
I'm curious about this. Go has a history of doing the simplest thing that covers the majority of cases, inherited from Plan 9. For one thing, although I don't know its current GC story, its original collector was very simple. I wonder what would happen if you transplanted a recent collector from Java, where the strategy is to optimize the crap out of anything that moves, into Go? Obviously, it wouldn't be as fast as not allocating, or at least not collecting, in hot code, but what would be the actual penalty? On the other hand, what about, say, Pony and its per-thread collectors?
- stmw 9y ago
  From a recent talk on the latest JDK GC, it seems to require even more tuning parameters than before. So the best performance achievable may be higher, but with more manual work. Which is along the same trade-off curve, with malloc/free on one end, and a very simple GC on the other.
tedunangst 9y ago
Graph complaint: the orange and blue lines arent labeled which is which.
twotwotwo 9y ago
FWIW, I think this mostly shows that the GOGC defaults will surprise you when your program keeps a small amount of live data but allocates quickly and is CPU bound. By default, it's trying to keep allocated heap size under 2x live data. If you have a tiny crypto bench program that keeps very little live data around, it won't take many allocations to trigger a GC. So the same code benchmarked here would trigger fewer GCs in the context of a program that had more data live. For example, it would have gone faster if the author had allocated a 100MB []byte in a global variable. If your program is far from exhausting the RAM but is fully utilizing the CPU, you might want it to save CPU by using more RAM, equivalent to increasing GOGC here. The rub is that it's hard for the runtime ever be sure what the humans want without your input: maybe this specific Go program is a little utility that you'd really like to use no more RAM than must to avoid interference with more important procs. Or maybe it's supposed to be the only large process on the machine, and should use a large chunk of all available RAM. Or it's in between, if there are five or six servers (or instances of a service) sharing a box. You can imagine heuristic controls that override GOGC in corner cases (e.g., assume it's always OK to use 1% of system memory), or even a separate knob for max RAM use you could use in place of GOGC. But the Go folks tend to want to keep things simple, so right now you sometimes have to play with GOGC values to get the behavior you want.
tapirl 9y ago
Using the SetGCPercent option with a large value is dangerous, it may cause a program never get GCed before virtual memory is exploited. And when this happens, Go programs will become very very slow. The current official Go runtime performs very very poorly when virtual memory is involved. It may make the while OS non-interactive so that you must restart your computer. A SetGCMemoryThreshold(maxMemory int) opition is better for many programs.
- twotwotwo 9y ago
  Definitely agree option of a memory limit would be handy in a lot of cases. Sometimes a box exists to run one service and you know you want like 75% of the box's RAM (leaving some for cache, other procs, etc.). Other times, like this, you don't want GOGC=100's frequent collections but also don't want the risk of an explosion if live data increases. Just adding some heuristics to override GOGC's behavior in extreme cases could help, e.g. assume by default OK to use 1% of system RAM and not OK to use >75% of it regardless of live data. I'm not sure we can realistically hope to get that--adds complexity, folks dislike arbitrary thresholds, and any change will break users who have optimized around current behavior. It might be you could approximate the desired behavior for reasonably well-behaved programs from a third-party package using runtime methods and a timer, but that's a drag.

272 points by jgrahamc 9y ago

This is a good illustration that Go isn't as simple as its proponents like to praise[1]. When you face complex engineering challenges, you need tools with a certain amount of complexity. [1] especially when comparing to Java when it comes to GC customization.

mining 9y ago
I don't know. That seemed like a fairly simple exploration of the problem and an easy library function to solve it. Would it be much simpler in any other language?
- dboreham 9y ago
  Simpler would be "just work like I expect".
  - zaarn 9y ago
    Doesn't work with GCs. As another comment in this thread explained, GCs have CPU and Memory tradeoffs and you can easily make a GC that would work excellent in the OP benchmark but would suffer severely under other workloads.
  - merlish 9y ago
    I think the expectation the Go lot have been working towards is the expectation that individual pauses are very short. Contrast to a Java server GC, aiming towards overall efficiency. (Tell me if I'm wrong, I'm no expert.)
    - weberc2 9y ago
      Yes, Go optimizes for low latency and Java for high throughput.
      - pjmlp 9y ago
        Depends on the Java GC being used.
        weberc2 9y ago
        Granted, but the stock GC is optimized for throughput.
        pjmlp 9y ago
        The stock GC on OpenJDK, there are other JVMs to choose from.
  - falcolas 9y ago
    What about the use cases where what you expect is not what I expect? GC is, as a sibling comment says, about compromises around memory safety.
  - weberc2 9y ago
    As another commented posted elsewhere, GC is all about tradeoffs, and for a govern set of GC tradeoffs there will always be pathological cases. Another GC might work out of the box for this workload, but perform very badly for a workload at which Go's GC excels. As others have already mentioned, the fact that he was so easily able to tune the GC is a testament to Go's simplicity. There are lots of good criticisms of Go to be had, but it's runtime is pretty remarkable, in my opinion.
- lmm 9y ago
  It's a simple problem with a simple fix, from a very simple benchmark - but it's already reaching the limits of how much tuning Go offers. If even such a basic benchmark needed one GC tuning knob, that seems suggestive that maybe a larger and more complex program really would need all the knobs that Java offers, the ones that Go claims are unnecessary. Or maybe not - maybe that one knob really can solve all the problems. But I really was surprised that anyone would ever need to actively configure a GC for such a simple use case, so I've learnt something at least.
  - weberc2 9y ago
    I think most pathological cases can be reduced to similarly simple examples. I'm sure these exist for other GCs as well.
    - stmw 9y ago
      Yes, the take-away here is not that Go is particularly bad, but that GCs are problematic. Notably, GC != automatic memory management.
      - littlestymaar 9y ago
        > Notably, GC != automatic memory management. Actually, GC == automatic memory management, just not automatic performant memory management
      - weberc2 9y ago
        That's not a reasonable conclusion to draw. You would need to preallocate to maximize performance regardless of GC in this case, and in other cases, the performance and determinism gaps between GC languages and non-GC languages is almost entirely due to tertiary factors. For example, GC languages are more likely to prioritize friendliness over raw performance in other areas--language features, compiler optimizations, standard library design, etc. Non-GC languages really only need to appeal to the high-performance and high-determinism crowds.
  - stmw 9y ago
lucozade 9y ago
Funny what different people get from these things. I was most impressed that he managed to get very close to perfect scalability by twiddling the one knob on offer. Java GC customisation is an art. This analysis was entirely mechanical. If the author wasn't just lucky i.e. that this is sufficient for a large number of workload profiles, then that's very good news.
- marcosdumay 9y ago
  The problem of software knobs is that the one that helps you is almost never there, and you usually can't get similar functionality from mixing the ones that are there. That said, there are exceptions. I don't think GC is something simple enough to be an exception, but those exist.
- tveita 9y ago
  The main problem is that it isn't a very good knob for this use case, as "ratio of freshly allocated data to live data remaining" is only incidental to the goal of the person writing the program. Therefore you'd expect to have to retune it due to unrelated changes, like if you allocated a big chunk of static memory to hold results. I expect Go's current behaviour is right for memory-intensive programs, where "limit the wasted memory as a ratio of the actual working set" is a reasonable high-level goal. For a CPU intensive program you might have preferred a "max x% CPU overhead" knob, but just a minimum memory size would work, as this is mostly an artifact of the very small memory usage of benchmark program. Java's G1 has "max pause delay" as their preferred knob, but Go has stuck that knob on low since around version 1.6, which is a good choice for what is essentially a language for network services.
merlish 9y ago
Interesting! IMO it showed that garbage collection isn't some silver bullet (of course), but that with a kick to a quite coarse control it's possible to achieve very good performance indeed. I am surprised that the difference is so great - and that the GC seems to think it's a good idea to keep collecting 4 MB of data at a time - but I guess the Go team have been tuning heavily for interactive performance i.e. small pauses. I guess in a situation like this you'd prefer something more like 'server' GC - maybe wait until you've allocated 1 GB then run through and collect everything you can. In games, I wonder if you'd like to turn the GC off during frame rendering then collect while presenting the frame/waiting for VSync. For soft real-time audio - synthesizing a few samples at a time - what would you do? OpenAL wants you to queue up thousands or tens of thousands of samples, but that sucks if you need to react very quickly. If anyone has any ideas, please say. I'd love to do soft real-time audio in a thread even in e.g. C#. (Edit: Maybe JIT some code that doesn't allocate? Or even have a separate interpreter written in C that has a constant-ish upper performance bound & no malloc.)
Thaxll 9y ago
Having worked with Java GC it's just a nightmare to tune. I prefer languages that have almost no options for GC tuning.
- rrampage 9y ago
  On Java 8+, you can use G1 GC which has very few knobs. On the other hand, CMS GC has a lot of knobs you can tune depending on your workload. JVM (Hotspot in this case) believes in flexibility over a one-size fits all. YMMV.
  - stmw 9y ago
    In other words, there are actually knobs that have you choose the GC, and then choose settings within the GC.
- erik_seaberg 9y ago
  Tuning is a hassle, but when the alternative is rewriting my important code over and over until I hit the right tradeoff between fast and readable, I'll take it.

jasode 9y ago

Whenever I see GC, I immediately think of tradeoffs. One can optimize a GC algorithm (or GC configuration setting) to work well for one type of workload. Likewise, one can always create a pathological workload for any GC configuration because GC has unavoidable tradeoff between cpu time and memory footprint. Therefore, I read his article with the intent of placing his findings in the "taxonomy" of GC tradeoffs. In that vein, I noticed that his benchmarks and graphs do not include measurements for memory footprint. Instead, it has throughput statistics such as "ops/sec" or "sign/second". I think I see why working memory size isn't emphasized. He's testing a crypto algorithm "ECDSA-P256 Sign". I'm not familiar with it but I assume we can think of it as a typical hash function[1] that doesn't require unpredictable dynamic allocation of memory as the algorithm processes the bytes of input. (A typical hash function uses the same fixed amount of memory whether hashing 1 kilobyte or 1 terabyte.) If his synthetic benchmark to play with GOGC was using a different type of workload... such as a text parser that stressed the GC with thousands of different sized memory objects, we'd see memory footprint as part of the analysis. [1] https://en.wikipedia.org/wiki/Elliptic_Curve_Digital_Signature_Algorithm https://en.wikipedia.org/wiki/Elliptic_Curve_Digital_Signatu...

dullgiulio 9y ago
It's not just a fixed-length hash, but it does computations on the curve points, which are represented by big.Int's, that take a variable size and are likely to end up on the heap. Also, Go crypto makes a heavy use of interface that also make stuff escape to the heap. Of such things are a bottleneck, it might be worthwhile to "un-interface" them.

dboreham 9y ago

This doesn't read like a "GC is slow" problem to me, more like a "GC doesn't cope well in the face of a highly concurrent allocation workload". That's pretty bad though. As soon as you go down the rabbit hole of tweaking knobs to cajole GC into doing what you expect, darkness lies beyond..

jchw 9y ago
It sounds like a bug in the GC for it to run so often, but even so, Go's GC has only one knob and they're adament about keeping it that way. There's no rabbit hole to fall into here.
- Twirrim 9y ago
  There can be advantages (and ZOMG so much confusion) in allowing multiple knobs though. Java takes it to an extreme and we're somewhat in the territory of needing a black box opitimiser, but you can adjust the GC to suit a really wide variety of workloads and requirements. By leaving it to one knob, Google/Go is heavily limiting the uses of the language.
jerf 9y ago
All GC algorithms have pessimal cases; I believe there is a mathematical proof of that, though I can never find it on the net through the chatter of all the other stuff about garbage collection. This is a pessimal case for the current Go GC, but that alone doesn't prove much because all algorithms have pessimal cases. (Even "manual memory management" can end up with pessimal cases for the automatic part, which is why you might see an improvement with your "manually managed" program by swapping malloc for jemalloc or something. What people call "manual memory management" still generally includes a lot of automatic stuff.) I believe Go does something like set a threshold of memory use. Once that memory usage is hit, a collection is run. After that collection is run, it uses the new accurate amount of memory used to set the next threshold. The threshold of course start small and have an exponential increase of some sort. When the program first starts, the threshold is set fairly low. If you immediately start allocating lots of things and keeping them, a common case, it grows to a sensible size in a reasonable number of allocations and sweeps (which as you are growing into your initial footprint, amortize out pretty cheap), and in a lot of cases you won't think much about the GC until you are really pushing performance. If, however, you write a program that isn't holding on to much RAM in the steady state, such as a crypto algorithm (which for all their algorithmic complexity typically don't take much RAM to run, as compared to, say, loading an image, and I don't know for sure but it's possible this ends up all stack-allocated), and that algorithm allocates a lot of short-term stuff, perhaps even just allocates for its final answer, then you get into a situation where you are quickly generating enough garbage to cross the initial low threshold, but the GC immediately cleans it all up when it runs, so when it is deciding on what the new threshold should be it still picks a low number, causing it to repeat the cycle indefinitely. Fixes will immediately leap to your mind, but remember you must evaluate your fixes holistically, not as if this is the only case in the world. If it were the only case in the world, it would be an easy problem, and Go never would have exhibited the problem in the first place. In particular, as you consider your "fix", consider also that a real program that used real crypto would be much less likely to see this pessimal case occur, because it is very likely the rest of the program would push the memory thresholds up to something sensible, so the fix of "do nothing" is actually quite likely to work well on non-artifical-benchmark programs. It is likely that the most obvious fix that comes to mind will result in making a program that is deliberately running some small server process or something appearing to leak memory after running for a long period of time, for instance. (I suspect quite strongly in this case that the correct "fix" is to do nothing.)

alkonaut 9y ago

If you make something that is performance critical, and in particular if it's concurrent, the normal rule is you avoid allocating on the heap at all. Use object preallocated data (just operate on massive lists), or use pools of data etc. Any language/runtime will support just allocating a large blob of data and then playing with it with code that looks like C. Effectively once you want to write perf sensitive code in a garbage collectied language you have to stop writing idiomatic code for that language. That can mean using data orientation (SoA for example is a very non-idiomatic thing to do in most C-like languages and especially OO ones). Not using heap allocation at all for any length of time is definitely non-idiomatic in Java/C#/Python/js etc., but that's what you need to do if you want any kind of performance. There are two truths here: 1) Any language can be as fast as C and 2) When they are as fast as C, they also look like C regardless what it was to begin with.

pjmlp 9y ago
> Effectively once you want to write perf sensitive code in a garbage collectied language you have to stop writing idiomatic code for that language. Only on GC languages that don't offer proper language features for value types. Sadly most of them failed to gain market share, like Modula-3. Eiffel and D are some of few around that offer such capabilities, and C# has been getting some love from System C# features, with spans, new ref types, and GC regions.
- alkonaut 9y ago
  Yes, C# is slowly getting there (I'm a desktop C# dev since 14 years so I have waited for these features to creep in). Value types certainly help, but in idiomatic C# if you foreach over something you (sometimes) allocate an iterator on the heap. And it's not clear from the syntax when you do. You just have to know that a List<T> doesn't heap allocate its enumerator but most other types do, for example. These details are mostly hidden from you. Using the low level features in C# (in the past structs, preferably in an AoS layout, then Span<T> etc tomorrow) will effectively make it a "Safe C". When you use those, you can't use idiomatic C (linq, heap allocs etc). You effectively agree to use a completely different subset of the language for your high perf tasks, and when you do, it doesn't look like the C# you have in the rest of your app.

jnordwick 9y ago

As anybody who has done high-performance, low-latency Java and had to deal with writing GC-less Java code, this is no surprise at all. In the Java world we at least had ways around it with things like sun.misc.unsafe, avoiding language features that allocate, pooling, etc. While some of those things can be done in Go, I don't know if all the techniques are possible. Without them, I'm not sure Go can really compete in the arena.

stmw 9y ago
Exactly right re: Java. You could just use Rust, and get automatic memory management without GC.
- abiox 9y ago
  One does not simply "just use Rust".
  - kibwen 9y ago
    Here, allow me to help: https://www.rust-lang.org/en-US/install.html https://www.rust-lang.org/en-US/install.html

dm319 9y ago

Can anyone explain to me why he got better performance with GC turned on (and optimised) rather than off?

tom_mellior 9y ago
Judicious garbage collection reuses memory you have already allocated, and for which there may already be entries in caches, page tables, garbage collector metadata, and whatnot. If you never garbage collect, the runtime has to repeatedly allocate new memory from the operating system, and all this additional metadata has to be juggled. If allocating new memory is a system call, that has costs. If you get lots of page faults because you keep using new pages, that has costs as well.
- emeraldd 9y ago
  It sounds like instead of turning off GC, the algorithm was changed to an "automatic" allocation/deallocation scheme that's closer to what you might see with manual memory management in a C like world. I wonder if the Go runtime has any optimizations to turn things into stack/static allocations in that scenario.
  - tom_mellior 9y ago
    I'm not a Go expert, but my understanding of the blog was that the algorithm or any other aspect of the code was not changed. Only the garbage collection frequency.
    - emeraldd 9y ago
      My comment was in reply to the parent's: > If you never garbage collect, the runtime has to repeatedly allocate new memory from the operating system ...
      - tom_mellior 9y ago
        Yeah, that was me. I still don't understand what you're saying. Who is changing what algorithm in what component? If you tell the Go runtime not to garbage collect but keep allocating memory, that memory has to come from somewhere. That somewhere is the operating system.

TazeTSchnitzel 9y ago

The Go team were very proud of their garbage collector that “virtually eliminates” (IIRC) GC pauses. I guess it does that at the cost of severe program slowdown in cases like this? It's all about trade-offs, and I don't trust people who think there's only one thing to optimise for.

kibwen 9y ago
While the Go GC is definitely no slouch when it comes to engineering, the messaging around it has not been entirely forthright: https://blog.plan99.net/modern-garbage-collection-911ef4f8bd8e https://blog.plan99.net/modern-garbage-collection-911ef4f8bd...
- geodel 9y ago
  This blog is more of a rant. Official blog announcements and marketing material are not the place to delve deeply into technical tradeoffs of designing GC. He compares it with Java which is rather ironic considering I have not seen Java official announcements talking about things not done. It would like saying 'We have not been able to deliver value types, module story is still half done, and we couldn't write an official http cient in whole development cycle in Java 9' in release announcement of Java 9. Also there is no data provided to show how bad is Go's GC compare to Java

tuyguntn 9y ago

If any Cloudflare engineers are here, why is Rust not chosen as main language for such use cases?

acdha 9y ago
I like Rust too but it's a newer language and the Cloudflare team has a history of using Go since it's targeted pretty squarely at their kind of work. In the case of doing benchmarks, it's especially unhelpful to make that suggestion since the point of a benchmark is to learn where the limits are rather than an invitation to port everything to a different language to avoid a problem which might remain theoretical forever.
- staticassertion 9y ago
  I don't think they're trying to suggest using rust, just curious why Cloudflare chose a language with a GC for workloads that apparently do not play well with GC. "We already used Go" is a totally valid reason. "It's rare (or as you say potentially never an issue) that we actually need to avoid GC" is another.
  - khc 9y ago
    the context of the benchmark is https://blog.cloudflare.com/arm-takes-wing/ https://blog.cloudflare.com/arm-takes-wing/ (Cloudflare engineer but I have nothing to do with the recent blog posts)
  - acdha 9y ago
    “why is Rust not chosen as main language” seemed like an unnecessarily accusatory way to express that idea, especially given that language choices are usually driven by a number of factors rather than a single benchmark. I've seen a lot of advocacy over the years like that which has seemed to inspire more backlash than adoption. I generally prefer something constructive in the tradition of showing a better way to do something — i.e. in this case, perhaps show a Rust crypto library which delivers better performance so people could judge whether that kind of thing is enough of a draw to be worth migrating or, in the case of Rust, taking advantage of its embedding characteristics.

kbart 9y ago

This is why we will still have C for times to come.

tebruno99 9y ago
If you can't figure out how to tune 1 documented option in Go to fix this issue caused by this one specific use case (which he did figure it out and it was resolved).. Then you're probably not using C either.
- kbart 9y ago
  "Then you're probably not using C either." In fact, I do - daily. I'm not stating that this exact issue is the key reason, but because C is very simple with not much of an abstraction layer between software and hardware, where most such (unexpected) performance issues arises. Yes, it might be a single flag to makes things run normal, but it might also take 1 week in production to find such "magic" bugs.

stmw 9y ago

This kind of tuning ends up being required for all languages that use garbage collection. The more you use it in production, the more you're doing meta programming with environment variables, command-line options and your allocation profiles. The problem is not so much some performance reduction, it's that the performance reduction is often variable and unexpected, as it is in this blog post.

readittwice 9y ago

I would love to see an in-depth analysis why Go's GC behaves this way for this specific benchmark in the first place. Just playing around with different values for `GOGC` seems like guess-work and could just paper over the underlying real issue. I know this probably requires to read the GC's source code, but that would've been certainly been very educational.

Arzh 9y ago

Tuning is part of the job if you need high performance. It can take many forms from code optimization, build flags, platform configurations, to even hardware tweeks. Most people hate this kinda stuff but some of us think it's fun.

mcguire 9y ago

I'm curious about this. Go has a history of doing the simplest thing that covers the majority of cases, inherited from Plan 9. For one thing, although I don't know its current GC story, its original collector was very simple. I wonder what would happen if you transplanted a recent collector from Java, where the strategy is to optimize the crap out of anything that moves, into Go? Obviously, it wouldn't be as fast as not allocating, or at least not collecting, in hot code, but what would be the actual penalty? On the other hand, what about, say, Pony and its per-thread collectors?

stmw 9y ago
From a recent talk on the latest JDK GC, it seems to require even more tuning parameters than before. So the best performance achievable may be higher, but with more manual work. Which is along the same trade-off curve, with malloc/free on one end, and a very simple GC on the other.

tedunangst 9y ago

Graph complaint: the orange and blue lines arent labeled which is which.

twotwotwo 9y ago

FWIW, I think this mostly shows that the GOGC defaults will surprise you when your program keeps a small amount of live data but allocates quickly and is CPU bound. By default, it's trying to keep allocated heap size under 2x live data. If you have a tiny crypto bench program that keeps very little live data around, it won't take many allocations to trigger a GC. So the same code benchmarked here would trigger fewer GCs in the context of a program that had more data live. For example, it would have gone faster if the author had allocated a 100MB []byte in a global variable. If your program is far from exhausting the RAM but is fully utilizing the CPU, you might want it to save CPU by using more RAM, equivalent to increasing GOGC here. The rub is that it's hard for the runtime ever be sure what the humans want without your input: maybe this specific Go program is a little utility that you'd really like to use no more RAM than must to avoid interference with more important procs. Or maybe it's supposed to be the only large process on the machine, and should use a large chunk of all available RAM. Or it's in between, if there are five or six servers (or instances of a service) sharing a box. You can imagine heuristic controls that override GOGC in corner cases (e.g., assume it's always OK to use 1% of system memory), or even a separate knob for max RAM use you could use in place of GOGC. But the Go folks tend to want to keep things simple, so right now you sometimes have to play with GOGC values to get the behavior you want.

tapirl 9y ago

Using the SetGCPercent option with a large value is dangerous, it may cause a program never get GCed before virtual memory is exploited. And when this happens, Go programs will become very very slow. The current official Go runtime performs very very poorly when virtual memory is involved. It may make the while OS non-interactive so that you must restart your computer. A SetGCMemoryThreshold(maxMemory int) opition is better for many programs.

twotwotwo 9y ago
Definitely agree option of a memory limit would be handy in a lot of cases. Sometimes a box exists to run one service and you know you want like 75% of the box's RAM (leaving some for cache, other procs, etc.). Other times, like this, you don't want GOGC=100's frequent collections but also don't want the risk of an explosion if live data increases. Just adding some heuristics to override GOGC's behavior in extreme cases could help, e.g. assume by default OK to use 1% of system RAM and not OK to use >75% of it regardless of live data. I'm not sure we can realistically hope to get that--adds complexity, folks dislike arbitrary thresholds, and any change will break users who have optimized around current behavior. It might be you could approximate the desired behavior for reasonably well-behaved programs from a third-party package using runtime methods and a timer, but that's a drag.