7 ms·

You really haven't given any supporting information for your argument other than a vague feeling that GC is somehow bad. In fact you just pointed out many count

by titzer 5y ago

You really haven't given any supporting information for your argument other than a vague feeling that GC is somehow bad. In fact you just pointed out many counterexamples to your own argument, so I'm not sure what to take away. I've seen this sentiment a lot, and I never see specifics. "GC is bad for systems language" is an unsupported, tribalist, firmly-held belief that is unsupported by hard data. On the other hand, huge, memory-intensive and garbage-collected systems have been deployed in vast numbers by thousands of different companies for decades, long before Go, within acceptable latency bounds. And shoddy, poorly performing systems have been written in C/C++ and failed spectacularly for all kinds of reasons.

throwaway894345 5y ago
Virtually invariably, "GC is bad" assumes (1) lots of garbage (2) long pause times. Go has idiomatic value types (so it generates much less garbage) and a low-latency garbage collector. People who argue against GC are almost always arguing against some Java GC circa 2005.
- MaulingMonkey 5y ago
  This is the no true scottsman argument. I mean, no true modern GC. And it's bullshit. Let's be topical and pick on Go since that's the language in the title: https://blog.twitch.tv/en/2019/04/10/go-memory-ballast-how-i-learnt-to-stop-worrying-and-love-the-heap/ https://blog.twitch.tv/en/2019/04/10/go-memory-ballast-how-i... 30% of CPU spent on GC, individual GC pauses already in the milliseconds, despite a tiny half-gig heap in 2019. For gamedev, a single millisecond in the wrong place can be enough to miss vsync and have unacceptable framerate stutter. In the right place, it's "merely" 10% of your entire frame budget, for VR-friendly, nausea-avoiding framerates near 100fps. Or perhaps 90% if "single digit milliseconds" might include 9ms. Meanwhile, the last professional project I worked on had 100ms pauses every 30 seconds because we were experimenting with duktape, which is still seeing active commits. Closer to a 32GB heap for that project, but most of that was textures. Explicit allocation would at least show where the problematic garbage/churn was in any profiler, but garbage collection meant a single opaque codepath for all garbage deallocation... without even the benefit of explicit static types to narrow down the problem.
  - throwaway894345 5y ago
    I don't think you know what "no true scotsman" means--I'm not asserting that Go's GC is the "true GC" but that it is one permutation of "GC" and it defies the conventional criticisms levied at GC. As such, it's inadequate to refute GC in general on the basis of long pauses and lots of garbage, you must refute each GC (or at least each type/class of GC) individually. Also, you can see how cherry-picking pathological, worst-case examples doesn't inform us about the normative case, right?
    - deleted 5y ago
      [deleted]
    - MaulingMonkey 5y ago
      >> And it's bullshit. > cherry picks worst-case examples and represents them as normative Neither of my examples are anywhere near worst-case. All texture data bypassed the GC entirely, for example, contributing to neither live object count nor GC pressure. I'm taking numbers from a modern GC with value types that you yourself should be fine and pointed out, hey, it's actually pretty not OK for anything that might touch the render loop in modern game development, even if it's not being used as the primary language GC.
    - MaulingMonkey 5y ago
      > I don't think you know what "no true scotsman" means--I'm not asserting that Go's GC is the "true GC" At no point in invoking https://en.wikipedia.org/wiki/No_true_Scotsman https://en.wikipedia.org/wiki/No_true_Scotsman does one bother to define what a true scotsman is, only what it is not by way of handwaving away any example of problems with a category by implying the category excludes them. It's exactly what you've done when you state "People who argue against GC are almost always arguing against" some ancient, nonmodern, unoptimized GC. Modern GCs have perf issues in some categories too. > As such, it's inadequate to refute GC in general on the basis of long pauses and lots of garbage, you must refute each GC (or at least each type/class of GC) individually. I do not intend to refute the value of GCs in general. I will happily use GCs in some cases. I intend to refute your overbroad generalization of the anti-GC camp, for which specific examples are sufficient. > Also, you can see how cherry-picking pathological, worst-case examples doesn't inform us about the normative case, right? My examples are neither pathological nor worst case. They need not be normative - but for what it's worth, they do exemplify the normative case of my own experiences in game development across multiple projects with different teams at different studios, when language level GCs were used for general purpouses, despite being bypassed for bulk data. It's also exactly what titzer was complaining was missing upthread: > I've seen this sentiment a lot, and I never see specifics. "GC is bad for systems language" is an unsupported, tribalist, firmly-held belief that is unsupported by hard data.
- throwaway684936 5y ago
  What do you consider a long pause time? In userland I consider anything above maybe 1 or 2 milliseconds to be a long pause time. The standards only get higher when it's something like a kernel. A kernel with even a 0.2ms pause time can be unacceptable when working with ultra low-latency audio for example.
  - throwaway894345 5y ago
    The criticisms I've heard typically reference pause times in the tens or hundreds of milliseconds. Agreed that different domains and applications have different requirements for pause times. I would be very interested to see histograms of pause times for different GCs, but I'm pretty sure the specific results would vary a lot depending on the specific corpus of applications under benchmark. If your 99%-ile pause time is tens of microseconds, is that good enough for video games? Audio kernels?
Karrot_Kream 5y ago
Right. In my experience just taking a few steps (like pre-allocating buffers or arrays) decrease GC pressure enough where GC runs don't actually affect performance enough to matter (as long as you're looking at ~0.5-1 ms P99 response times). But there's always the strident group who says GCs are bad and never offer any circumstance where that could be true.
- titzer 5y ago
  Indeed. What really kills is extremely high allocation rates and extremely high garbage production. I've seen internal numbers from $megacorp that show that trashy C++ programs (high allocation + deallocation rates) look pretty much the same to CPUs as trashy Java programs, but are far worse in terms of memory fragmentation. Trashy C++ programs can end up spending 20+% of their execution time in malloc/free. That's a fleetwide number I've seen on clusters > 1M cores. I will admit that the programming culture is different for many GC'd languages' communities, sometimes encouraging a very trashy programming style, which contributes to the perception that GC itself is the problem, but based on my experience in designing and building GC'd systems, I don't blame GC itself.
  - ninkendo 5y ago
    > I will admit that the programming culture is different for many GC'd languages' communities, sometimes encouraging a very trashy programming style, which contributes to the perception that GC itself is the problem For some languages (I’m looking at you, Java), there’s not much of a way to program that doesn't generate a bunch of garbage, because only primitives are treated as value types, and for Objects, heap allocations can only be avoided if escape analysis can prove the object doesn’t outlast its stack frame (which isn’t reliable in practice.) (Edit: or maybe it doesn’t happen at all. Apparently escape analysis isn’t used to put objects on the stack even if they are known to not escape: https://www.beyondjava.net/escape-analysis-java https://www.beyondjava.net/escape-analysis-java) I honestly can’t imagine much of a way to program in Java that doesn’t result in tremendous GC pressure. You could technically allocate big static buffers and use a completely different paradigm where every function is static and takes data at a defined offsets to said buffers, but… nobody really does this and the result wouldn’t look anything like Java. Sometimes it’s appropriate to blame the language.
fauigerzigerk 5y ago
>I've seen this sentiment a lot, and I never see specifics. "GC is bad for systems language" is an unsupported, tribalist, firmly-held belief that is unsupported by hard data. I would argue it's not (very) hard data that we need in this case. My opinion is that the resource usage of infrastructure code should be as low as possible so that most resources are available to run applications. The economic viability of application development is very much determined by developer productivity. Many applications aren't even run that often if you think of in-house business software for instance. So application development is where we have to spend our resource budget. Systems/infrastructure code on the other hand is subject to very different economics. It runs all the time. The ratio of development time to runtime is incredibly small. We should optimise the heck out of infrastructure code to drive down resource usage whenever possible. GC has significant memory and CPU overhead. I don't want to spend double digit resource percentages on GC for software that could be written differently without being uneconomical.
- titzer 5y ago
  > Systems/infrastructure code on the other hand is subject to very different economics. It runs all the time. The ratio of development time to runtime is incredibly small. We should optimise the heck out of infrastructure code to drive down resource usage whenever possible. I will assume by "infrastructure code" you mean things like kernels and network stacks. Unfortunately there are several intertwined issues here. First, we pay in completely different ways for writing this software in manually-managed languages. Security vulnerabilities. Bugs. Development time. Slow evolution. I don't agree with the tradeoffs we have made. This software is important and needs to be memory-safe. Maybe Rust will deliver, who knows. But we currently have a lot of latent memory management bugs here that have consistently clocked in 2/3 to 3/4 of critical CVEs over several decades. That's a real problem. We aren't getting this right. Second, infrastructure code does not consume a lot of memory. Infrastructure code mostly manages memory and buffers. The actual heap footprint of the Linux kernel is pretty small; it mostly indexes and manages memory, buffers, devices, packets, etc. That is where optimization should go; manage the most resources with the lowest overhead just in terms of data structures. > GC has significant memory and CPU overhead. I don't want to spend double digit resource percentages on GC for software that could be written differently without being uneconomical. Let's posit 20% CPU for all the things that a GC does. And let's posit 2X for enough heap room to keep the GC running concurrently well enough that it doesn't incur a lot of mutator pauses. If all that infrastructure is taking 10% of CPU and 10% of memory, we are talking adding 2% CPU and 10% memory. ABSOLUTE BARGAIN in my book! The funny thing is, that people made these same arguments back in the heyday of Moore's law when we were getting 2X CPU performance ever 18 months. 2% CPU back then was a matter of weeks of Moore's law. Now? Maybe a couple of months. We consistently choose to spend the performance dividends of hardware performance improvements on...more performance? And nothing on safety or programmability? I seriously think we chose poorly here due to some significant confusion in priorities and real costs.
osigurdson 5y ago
GC is bad if the problem domain you are working on requires you to think about the behaviour of the GC all the time. GC behaviour can be subtle, may change from version to version and some behaviours may not even be clearly documented. If one has to think about it all the time, it is likely better just to use a tool where memory management is more explicit. However, I think for many examples of "systems software" (Kubernetes for example), GC is not an issue at all but for others it is an illogical first choice (though it often can be made to work).
- throwaway894345 5y ago
  Even in performance-critical software, you're not thinking about GC "all the time" but only in certain hot paths. Also, value types and some allocation semantics (which Go technically lacks, but the stack analyzer is intuitive and easily profiled so it has the same effect as semantics) make the cognitive GC burden much lower.
emtel 5y ago
My argument against GC (and which applies similary to JIT-basd runtimes) is that the problems caused by GC pauses have non-local causes. If a piece of code ran slowly because of a GC pause, the cause of the pause is in some sense _the entire rest of the system_. You can't fix the problem with a localized change. Programs in un-managed languages can be slow too, and excessive use of malloc() is a frequent culprit. But the difference is that if I have a piece of code that is slow because it is calling malloc() too much, I can often (or at least some of the time) just remove the malloc() calls from that function. I don't have to boil the ocean and significantly reduce the rate at which my entire program allocates memory. I think another factor that gets ignored is how much you care about tail latency. I think GC is usually fine for servers and other situations where you are targeting a good P99 or P99.9 latency number. And indeed, this is where JVM, Go, node.js, and other GCed runtimes dominate. But, there are situations, like games, where a bad P99.9 frame time means dropping a frame every 15 seconds (at 60fps). If you've got one frame skip every 10 seconds because of garbage collection pauses and you want to get to one frame skip every minute, that is _not_ an easy problem to fix. (Yes, I am aware that many commercial game engines have garbage collectors).
- bcrosby95 5y ago
  I don't want to try to bring up an exception that disproves your rule, but what about something like BEAM, where it has per-process (process = lightweight thread) heaps and GC.
  - emtel 5y ago
    I don't know anything about BEAM, but I don't think single-threading of any form really addresses the underlying problem. If you go to allocate something, and the system decides a GC is necessary in order to satisfy your allocation, then the GC has to run before your allocation returns.
    - rbranson 5y ago
      You can't share objects across threads (called "processes") in BEAM, so it's very different. The GC only ever needs to pause one call stack at a time to do a full GC cycle. Memory shared across processes is generally manually managed, typically more akin to a database than an object heap.

by titzer 5y ago

throwaway894345 5y ago

Virtually invariably, "GC is bad" assumes (1) lots of garbage (2) long pause times. Go has idiomatic value types (so it generates much less garbage) and a low-latency garbage collector. People who argue against GC are almost always arguing against some Java GC circa 2005.

MaulingMonkey 5y ago
This is the no true scottsman argument. I mean, no true modern GC. And it's bullshit. Let's be topical and pick on Go since that's the language in the title: https://blog.twitch.tv/en/2019/04/10/go-memory-ballast-how-i-learnt-to-stop-worrying-and-love-the-heap/ https://blog.twitch.tv/en/2019/04/10/go-memory-ballast-how-i... 30% of CPU spent on GC, individual GC pauses already in the milliseconds, despite a tiny half-gig heap in 2019. For gamedev, a single millisecond in the wrong place can be enough to miss vsync and have unacceptable framerate stutter. In the right place, it's "merely" 10% of your entire frame budget, for VR-friendly, nausea-avoiding framerates near 100fps. Or perhaps 90% if "single digit milliseconds" might include 9ms. Meanwhile, the last professional project I worked on had 100ms pauses every 30 seconds because we were experimenting with duktape, which is still seeing active commits. Closer to a 32GB heap for that project, but most of that was textures. Explicit allocation would at least show where the problematic garbage/churn was in any profiler, but garbage collection meant a single opaque codepath for all garbage deallocation... without even the benefit of explicit static types to narrow down the problem.
- throwaway894345 5y ago
  I don't think you know what "no true scotsman" means--I'm not asserting that Go's GC is the "true GC" but that it is one permutation of "GC" and it defies the conventional criticisms levied at GC. As such, it's inadequate to refute GC in general on the basis of long pauses and lots of garbage, you must refute each GC (or at least each type/class of GC) individually. Also, you can see how cherry-picking pathological, worst-case examples doesn't inform us about the normative case, right?
  - deleted 5y ago
    [deleted]
  - MaulingMonkey 5y ago
    >> And it's bullshit. > cherry picks worst-case examples and represents them as normative Neither of my examples are anywhere near worst-case. All texture data bypassed the GC entirely, for example, contributing to neither live object count nor GC pressure. I'm taking numbers from a modern GC with value types that you yourself should be fine and pointed out, hey, it's actually pretty not OK for anything that might touch the render loop in modern game development, even if it's not being used as the primary language GC.
  - MaulingMonkey 5y ago
    > I don't think you know what "no true scotsman" means--I'm not asserting that Go's GC is the "true GC" At no point in invoking https://en.wikipedia.org/wiki/No_true_Scotsman https://en.wikipedia.org/wiki/No_true_Scotsman does one bother to define what a true scotsman is, only what it is not by way of handwaving away any example of problems with a category by implying the category excludes them. It's exactly what you've done when you state "People who argue against GC are almost always arguing against" some ancient, nonmodern, unoptimized GC. Modern GCs have perf issues in some categories too. > As such, it's inadequate to refute GC in general on the basis of long pauses and lots of garbage, you must refute each GC (or at least each type/class of GC) individually. I do not intend to refute the value of GCs in general. I will happily use GCs in some cases. I intend to refute your overbroad generalization of the anti-GC camp, for which specific examples are sufficient. > Also, you can see how cherry-picking pathological, worst-case examples doesn't inform us about the normative case, right? My examples are neither pathological nor worst case. They need not be normative - but for what it's worth, they do exemplify the normative case of my own experiences in game development across multiple projects with different teams at different studios, when language level GCs were used for general purpouses, despite being bypassed for bulk data. It's also exactly what titzer was complaining was missing upthread: > I've seen this sentiment a lot, and I never see specifics. "GC is bad for systems language" is an unsupported, tribalist, firmly-held belief that is unsupported by hard data.
throwaway684936 5y ago
What do you consider a long pause time? In userland I consider anything above maybe 1 or 2 milliseconds to be a long pause time. The standards only get higher when it's something like a kernel. A kernel with even a 0.2ms pause time can be unacceptable when working with ultra low-latency audio for example.
- throwaway894345 5y ago
  The criticisms I've heard typically reference pause times in the tens or hundreds of milliseconds. Agreed that different domains and applications have different requirements for pause times. I would be very interested to see histograms of pause times for different GCs, but I'm pretty sure the specific results would vary a lot depending on the specific corpus of applications under benchmark. If your 99%-ile pause time is tens of microseconds, is that good enough for video games? Audio kernels?

Karrot_Kream 5y ago

Right. In my experience just taking a few steps (like pre-allocating buffers or arrays) decrease GC pressure enough where GC runs don't actually affect performance enough to matter (as long as you're looking at ~0.5-1 ms P99 response times). But there's always the strident group who says GCs are bad and never offer any circumstance where that could be true.

titzer 5y ago
Indeed. What really kills is extremely high allocation rates and extremely high garbage production. I've seen internal numbers from $megacorp that show that trashy C++ programs (high allocation + deallocation rates) look pretty much the same to CPUs as trashy Java programs, but are far worse in terms of memory fragmentation. Trashy C++ programs can end up spending 20+% of their execution time in malloc/free. That's a fleetwide number I've seen on clusters > 1M cores. I will admit that the programming culture is different for many GC'd languages' communities, sometimes encouraging a very trashy programming style, which contributes to the perception that GC itself is the problem, but based on my experience in designing and building GC'd systems, I don't blame GC itself.
- ninkendo 5y ago
  > I will admit that the programming culture is different for many GC'd languages' communities, sometimes encouraging a very trashy programming style, which contributes to the perception that GC itself is the problem For some languages (I’m looking at you, Java), there’s not much of a way to program that doesn't generate a bunch of garbage, because only primitives are treated as value types, and for Objects, heap allocations can only be avoided if escape analysis can prove the object doesn’t outlast its stack frame (which isn’t reliable in practice.) (Edit: or maybe it doesn’t happen at all. Apparently escape analysis isn’t used to put objects on the stack even if they are known to not escape: https://www.beyondjava.net/escape-analysis-java https://www.beyondjava.net/escape-analysis-java) I honestly can’t imagine much of a way to program in Java that doesn’t result in tremendous GC pressure. You could technically allocate big static buffers and use a completely different paradigm where every function is static and takes data at a defined offsets to said buffers, but… nobody really does this and the result wouldn’t look anything like Java. Sometimes it’s appropriate to blame the language.

fauigerzigerk 5y ago

>I've seen this sentiment a lot, and I never see specifics. "GC is bad for systems language" is an unsupported, tribalist, firmly-held belief that is unsupported by hard data. I would argue it's not (very) hard data that we need in this case. My opinion is that the resource usage of infrastructure code should be as low as possible so that most resources are available to run applications. The economic viability of application development is very much determined by developer productivity. Many applications aren't even run that often if you think of in-house business software for instance. So application development is where we have to spend our resource budget. Systems/infrastructure code on the other hand is subject to very different economics. It runs all the time. The ratio of development time to runtime is incredibly small. We should optimise the heck out of infrastructure code to drive down resource usage whenever possible. GC has significant memory and CPU overhead. I don't want to spend double digit resource percentages on GC for software that could be written differently without being uneconomical.

titzer 5y ago
> Systems/infrastructure code on the other hand is subject to very different economics. It runs all the time. The ratio of development time to runtime is incredibly small. We should optimise the heck out of infrastructure code to drive down resource usage whenever possible. I will assume by "infrastructure code" you mean things like kernels and network stacks. Unfortunately there are several intertwined issues here. First, we pay in completely different ways for writing this software in manually-managed languages. Security vulnerabilities. Bugs. Development time. Slow evolution. I don't agree with the tradeoffs we have made. This software is important and needs to be memory-safe. Maybe Rust will deliver, who knows. But we currently have a lot of latent memory management bugs here that have consistently clocked in 2/3 to 3/4 of critical CVEs over several decades. That's a real problem. We aren't getting this right. Second, infrastructure code does not consume a lot of memory. Infrastructure code mostly manages memory and buffers. The actual heap footprint of the Linux kernel is pretty small; it mostly indexes and manages memory, buffers, devices, packets, etc. That is where optimization should go; manage the most resources with the lowest overhead just in terms of data structures. > GC has significant memory and CPU overhead. I don't want to spend double digit resource percentages on GC for software that could be written differently without being uneconomical. Let's posit 20% CPU for all the things that a GC does. And let's posit 2X for enough heap room to keep the GC running concurrently well enough that it doesn't incur a lot of mutator pauses. If all that infrastructure is taking 10% of CPU and 10% of memory, we are talking adding 2% CPU and 10% memory. ABSOLUTE BARGAIN in my book! The funny thing is, that people made these same arguments back in the heyday of Moore's law when we were getting 2X CPU performance ever 18 months. 2% CPU back then was a matter of weeks of Moore's law. Now? Maybe a couple of months. We consistently choose to spend the performance dividends of hardware performance improvements on...more performance? And nothing on safety or programmability? I seriously think we chose poorly here due to some significant confusion in priorities and real costs.

osigurdson 5y ago

GC is bad if the problem domain you are working on requires you to think about the behaviour of the GC all the time. GC behaviour can be subtle, may change from version to version and some behaviours may not even be clearly documented. If one has to think about it all the time, it is likely better just to use a tool where memory management is more explicit. However, I think for many examples of "systems software" (Kubernetes for example), GC is not an issue at all but for others it is an illogical first choice (though it often can be made to work).

throwaway894345 5y ago
Even in performance-critical software, you're not thinking about GC "all the time" but only in certain hot paths. Also, value types and some allocation semantics (which Go technically lacks, but the stack analyzer is intuitive and easily profiled so it has the same effect as semantics) make the cognitive GC burden much lower.

emtel 5y ago

My argument against GC (and which applies similary to JIT-basd runtimes) is that the problems caused by GC pauses have non-local causes. If a piece of code ran slowly because of a GC pause, the cause of the pause is in some sense _the entire rest of the system_. You can't fix the problem with a localized change. Programs in un-managed languages can be slow too, and excessive use of malloc() is a frequent culprit. But the difference is that if I have a piece of code that is slow because it is calling malloc() too much, I can often (or at least some of the time) just remove the malloc() calls from that function. I don't have to boil the ocean and significantly reduce the rate at which my entire program allocates memory. I think another factor that gets ignored is how much you care about tail latency. I think GC is usually fine for servers and other situations where you are targeting a good P99 or P99.9 latency number. And indeed, this is where JVM, Go, node.js, and other GCed runtimes dominate. But, there are situations, like games, where a bad P99.9 frame time means dropping a frame every 15 seconds (at 60fps). If you've got one frame skip every 10 seconds because of garbage collection pauses and you want to get to one frame skip every minute, that is _not_ an easy problem to fix. (Yes, I am aware that many commercial game engines have garbage collectors).

bcrosby95 5y ago
I don't want to try to bring up an exception that disproves your rule, but what about something like BEAM, where it has per-process (process = lightweight thread) heaps and GC.
- emtel 5y ago
  I don't know anything about BEAM, but I don't think single-threading of any form really addresses the underlying problem. If you go to allocate something, and the system decides a GC is necessary in order to satisfy your allocation, then the GC has to run before your allocation returns.
  - rbranson 5y ago
    You can't share objects across threads (called "processes") in BEAM, so it's very different. The GC only ever needs to pause one call stack at a time to do a full GC cycle. Memory shared across processes is generally manually managed, typically more akin to a database than an object heap.