7 ms·

The article is from a database company, so I'll assume that approximates the scope. My scope for the GC discussion would include other parts that could be consi

The article is from a database company, so I'll assume that approximates the scope. My scope for the GC discussion would include other parts that could be considered similar software: cluster-control plane (Kubernetes), other databases, and possibly the first level of API services to implement a service like an internal users/profiles or auth endpoints. The tricky thing is GC works most of the time, but if you are working at scale you really can't predict user behavior, and so all of those GC-tuning parameters that were set six months ago no longer work properly. A good portion of production outages are likely related to cascading failures due to too long GC pauses, and a good portion of developer time is spent testing and tuning GC parameters. It is easier to remove and/or just not allow GC languages at these levels in the first place. On the other hand IMO GC-languages at the frontend level are OK since you'd just need to scale horizontally.

coder543 5y ago
Go doesn’t offer a bunch of GC tuning parameters. Really only one parameter, so your concerns about complex GC tuning here seem targeted at some other language like Java. This is a drawback in some cases, since one size never truly fits all, but it dramatically simplifies things for most applications, and the Go GC has been tuned for many years to work well in most places where Go is commonly used. The developers of Go continue to fix shortcomings that are identified. Go’s GC prioritizes very short STWs and predictable latency, instead of total GC throughput, and Go makes GC throughput more manageable by stack allocating as much as it can to reduce GC pressure. Generally speaking, Go is also known for using very little memory compared to Java.
- astrange 5y ago
  Is go's GC not copying/generational? I think "stack allocation" doesn't really make sense in a generational GC, as everything sort of gets stack allocated. Of course, compile-time lifetime hints might still be useful somehow.
  - coder543 5y ago
    > Is go's GC not copying/generational? Nope, Go does not use a copying or generational GC. Go uses a concurrent mark and sweep GC. Even then, generational GCs are not as cheap as stack allocation.
    - astrange 5y ago
      Stack's just the youngest generation. I can see the difference being that you have to scan a generation but the entire stack can be freed at once, but it still seems like an overly specific term. The general term elsewhere for multiple allocations you can free at the same time is "arenas".
      - coder543 5y ago
        From a conceptual point of view, I agree, but... in practice, stacks are incredibly cheap. The entire set of local variables can be allocated with a single bump of the stack pointer upon entry into a function, and they can all be freed with another bump of the stack pointer upon exit. With heap allocations, even with the simplest bump allocator.. you still have to allocate once per object, which can easily be an order of magnitude more work than what you have to do with an equivalent number of stack allocated objects. Your program doesn't magically know where those objects are, so it still also has to pay to stack allocate the pointers to keep track of the heap objects. Then you have the additional pointer-chasing slowing things down and the decreased effectiveness of the local CPU caches due to the additional level of indirection. A contiguous stack frame is a lot more likely to stay entirely within the CPU cache than a bunch of values scattered around the heap. Beyond that, and beyond the additional scanning you already mentioned, in the real world the heap is shared between all threads, which means there will be some amount of contention whenever you have to interact with the heap, although this is amortized some with TLABs (thread-local allocation buffers). You also have to consider the pointer rewriting that a generational GC will have to perform for the survivors of each generation, and that will not be tied strictly to function frames. The GC will run whenever it feels like it, so you may pay the cost of pointer rewriting for objects that are only used until the end of this function, just due to the coincidence of when the GC started working. I think (but could be wrong/outdated) that generational GCs almost all require both read and write barriers that perform some synchronization any time you interact with a heap allocated value, and this slows the program down even more compared to stack objects. (I believe that non-copying GCs don't need as many barriers, and that the barriers are only active during certain GC phases, which is beneficial to Go programs, but again, stack objects don't need any barriers at all, ever.) GCs are really cool, but stack allocated values are always better when they can be used. There's a reason that C# makes developer-defined value types available; they learned from the easily visible problems that Java has wrestled with caused by not allowing developers to define their own value types. Go took it a step further and got rid of developer-defined reference types altogether, so everything is a value type (arguably with the exception of the syntax sugar for the built-in map, slice, channel, and function pointer types), and even values allocated behind a pointer will still be stack allocated if escape analysis proves it won't cause problems.
- socialdemocrat 5y ago
  Java _needs_ lots of GC tuning parameters because you have practically no way of tuning the way your memory is used and organized in Java code. In Go you can actually do that. You can decide how data structures are nested, you can take pointers to the inside of a a block of memory. You could make e.g. a secondary allocator, allocating objects from a contiguous block of memory. Java doesn't allow those things, and thus it must instead give you lots of levers to pull on to tune the GC. It is just a different strategy of achieving the same thing: https://itnext.io/go-does-not-need-a-java-style-gc-ac99b8d26c60 https://itnext.io/go-does-not-need-a-java-style-gc-ac99b8d26...
  - native_samples 5y ago
    That's the Go party line but not really true. Counter-example: The Go GC is tuned for HTTP servers at latency sensitive companies like Google. It therefore prioritizes latency over throughput to an astonishing degree, which means it is extremely bad at batch jobs - like compilers. What language is the Go compiler written in? Go. This isn't fixable by simply writing the code differently. What you're talking about is in the limit equivalent to not using a GCd language at all, and you can do that with Java too via the Unsafe allocators. But it's not a great idea to do that too much, because then you may as well just bite the bullet and write C++. Java doesn't actually need lots of GC tuning parameters. Actually most of the time you can ignore them, because the defaults balance latency and throughput for something reasonable for the vast majority of companies that aren't selling ad clicks. But, if you want, you can tell the JVM more about your app to get better results like whether it's latency or throughput sensitive. The parameters are there mostly to help people with unusual or obscure workloads where Go simply gives up and says "if you have this problem, Go is not for you".
    - delian66 5y ago
      > it is extremely bad at batch jobs - like compilers. > What language is the Go compiler written in? Go. I do not see what are you trying to say? The Go compiler is plenty fast in my experience, especially compared to say `javac`. The startup time of `javac` (and most java programs) is atrocious.
- no_circuit 5y ago
  Yes, my comments were targeted to Java and Scala. Java has paid the bills for me for many years. I'd use Java for just about anything except for high load infrastructure systems. And if you're in, or want to be in, that situation, then why risk finding out two years later that a GC-enabled app is suboptimal? I'd guess you'd have no choice if in order to hire developers, you had to choose a language that the people found fun to use.
apalmer 5y ago
> A good portion of production outages are likely related to cascading failures due to too long GC pauses, and a good portion of developer time is spent testing and tuning GC parameters. Can’t really accept that without some kind of quantitative evidence.
- no_circuit 5y ago
  No worries. It is not meant to be quantitative. For a few years of my career that has been my experience. For this type of software, if I'm making the decision on what technology to use, it won't be any GC-based language. I'd rather not rely on promises that GC works great, or is very tunable. One could argue that I could just tune my services from time to time. But I'd just reduce the surface area for problems by not relying upon it at all -- both a technical and a business decision.
  - throwawaylala1 5y ago
    If you're needing to fight the GC to prevent crashes or whatever then you have a system design issue not a tooling/language/ecosystem issue. There are exceptions to this but they're rare and not worth mentioning in a broad discussion like this. Sadly very few people take interest in learning how to design systems properly. Instead they find comfort in tools that allow them to over-engineer the problems away. Like falling into zealotry on things like FP, zero-overhead abstractions, "design patterns", containerization, manual memory management, etc, etc. These are all nice things when properly applied in context but they're not a substitute for making good system design decisions. Good system design starts with understanding what computers are good at and what they suck at. That's a lot more difficult than it sounds because today's abstractions try to hide what computers suck at. Example: Computers suck at networking. We have _a lot_ of complex layers to help make it feel somewhat reliable. But as a fundamental concept, it sucks. The day you network two computers together is the day you've opened yourself up to a world of hurt (think race conditions) - so, like, don't do it if you don't absolutely have to.
    - Karrot_Kream 5y ago
throwaway894345 5y ago
> The tricky thing is GC works most of the time, but if you are working at scale you really can't predict user behavior, and so all of those GC-tuning parameters that were set six months ago no longer work properly. A good portion of production outages are likely related to cascading failures due to too long GC pauses, and a good portion of developer time is spent testing and tuning GC parameters. It is easier to remove and/or just not allow GC languages at these levels in the first place. Getting rid of the GC doesn't absolve you of the problem, it just means that rather than tuning GC parameters, you've encoded usage assumptions in thousands of places scattered throughout your code base.
initplus 5y ago
It's impossible to spend any time tuning Go's GC parameters as they intentionally do not provide any. Go's GC is optimized for latency, it doesn't see the same kind of 1% peak latency issues you get in languages with a long tail of high latency pauses. Also consider API design - Java API (both in standard & third party libs) tend to be on the verbose side and build complex structures out of many nested objects. Most Go applications will have less nesting depth so it's inherently an easier GC problem. System designs that rely on allocating a huge amount of memory to a single process exist in a weird space - big enough that perf is really important, but small enough that single-process is still a viable design. Building massive monoliths that allocate hundreds of Gb's at peak load just doesn't seem "in vogue" anymore. If you are building a distributed system keeping any individual processes peak allocation to a reasonable size is almost automatic.
- erik_seaberg 5y ago
  You tune Go’s GC by rewriting your code. It’s like turning a knob but slower and riskier.
  - coder543 5y ago
    You tune GC in Go by profiling allocations, CPU, and memory usage. Profiling shows you where the problems are, and Go has some surprisingly nice profiling tools built in. Unlike turning a knob, which has wide reaching and unpredictable effects that may cause problems to just move around from one part of your application to another, you can address the actual problems with near-surgical precision in Go. You can even add tests to the code to ensure that you're meeting the expected number of allocations along a certain code path if you need to guarantee against regressions... but the GC is so rarely the problem in Go compared to Java, it's just not something to worry about 99% of the time. If knobs had a "fix the problem" setting, they would already be set to that value. Instead, every value is a trade off, and since you have hundreds of knobs, you're playing an impossible optimization game with hundreds of parameters to try to find the set of parameter values that make your entire application perform the way you want it to. You might as well have a meta-tuner that just randomly turns the knobs to collect data on all the possible combinations of settings... and just hope that your next code change doesn't throw all that hard work out the window. Go gives you the tools to tune different parts of your code to behave in ways that are optimal for them. It's worth pointing out that languages like Rust and C++ also require you to tune allocations and deallocations... this is not strictly a GC problem. In those languages, like in Go, you have to address the actual problems instead of spinning knobs and hoping the problem goes away. The one time I have actually run up against Go's GC when writing code that was trying to push the absolute limits of what could be done on a fleet of rather resource constrained cloud instances, I wished I was writing Rust for this particular problem... I definitely wasn't wishing I could be spinning Java's GC knobs. But, I was still able to optimize things to work in Go the way I needed them to even in that case, even if the level of control isn't as granular as Rust would have provided.
EdwardDiego 5y ago
> A good portion of production outages are likely related to cascading failures due to too long GC pauses, and a good portion of developer time is spent testing and tuning GC parameters After 14 years in JVM dev in areas where latency and reliability are business critical, I disagree. Yes, excessive GC stop the world pauses can cause latency spikes, and excessive GC time is bad, and yes, when a new GC algorithm is released that you think might offer improvements, you test it thoroughly to determine if it's better or worse for your workload. But a "good portion" of outages and developer time? Nope. Most outages occur for the same old boring reasons - someone smashed the DB with an update that hits a pathological case and deadlocks processes using the same table, a DC caught fire, someone committed code with a very bad logical bug, someone considered a guru heard that gRPC was cool and used it without adequate code review and didn't understand that gRPC's load balancing defaults to pick first, etc. etc. The outages caused by GC were very very few. Outages caused by screw-ups or lack of understanding of subtleties of a piece of tech, as common as they are in every other field of development. Then there's the question of what outages GCed languages _don't_ suffer. I've never had to debug corrupted memory, or how a use after free bug let people exfiltrate data.
- zinekeller 5y ago
  > I've never had to debug corrupted memory You're lucky! When OpenJDK was still closed-sourced Hotspot from Sun, we have chased bugs that Sun confirmed was a defect on how Hotspot handle memory (and this is on a ECC'd system of course), although these days I can't remind of anything remotely related. > or how a use after free bug let people exfiltrate data. Technically you're just outsourcing it :)
  - Karrot_Kream 5y ago
    > You're lucky! When OpenJDK was still closed-sourced Hotspot from Sun, we have chased bugs that Sun confirmed was a defect on how Hotspot handle memory (and this is on a ECC'd system of course), although these days I can't remind of anything remotely related. I mean sure. I remember having similar issues with early (< 2.3) Python builds as well. But in the last decade of my career, only a handful of outages were caused by Java GC issues. Most of them happened for a myriad of other architectural reasons.
exdsq 5y ago
I think I toggled with the GC for less than a week in my eight years experience including some systems stuff - maybe this is true at FANG scale but not for me!
eudoxus 5y ago
As many have replied, the available levers for 'GC-tuning' in go is almost non-existent. However, what we do have influence on is "GC Pressure" which is a very important metric we can move in the right direction if the application requires it.

coder543 5y ago

Go doesn’t offer a bunch of GC tuning parameters. Really only one parameter, so your concerns about complex GC tuning here seem targeted at some other language like Java. This is a drawback in some cases, since one size never truly fits all, but it dramatically simplifies things for most applications, and the Go GC has been tuned for many years to work well in most places where Go is commonly used. The developers of Go continue to fix shortcomings that are identified. Go’s GC prioritizes very short STWs and predictable latency, instead of total GC throughput, and Go makes GC throughput more manageable by stack allocating as much as it can to reduce GC pressure. Generally speaking, Go is also known for using very little memory compared to Java.

astrange 5y ago
Is go's GC not copying/generational? I think "stack allocation" doesn't really make sense in a generational GC, as everything sort of gets stack allocated. Of course, compile-time lifetime hints might still be useful somehow.
- coder543 5y ago
  > Is go's GC not copying/generational? Nope, Go does not use a copying or generational GC. Go uses a concurrent mark and sweep GC. Even then, generational GCs are not as cheap as stack allocation.
  - astrange 5y ago
    Stack's just the youngest generation. I can see the difference being that you have to scan a generation but the entire stack can be freed at once, but it still seems like an overly specific term. The general term elsewhere for multiple allocations you can free at the same time is "arenas".
    - coder543 5y ago
      From a conceptual point of view, I agree, but... in practice, stacks are incredibly cheap. The entire set of local variables can be allocated with a single bump of the stack pointer upon entry into a function, and they can all be freed with another bump of the stack pointer upon exit. With heap allocations, even with the simplest bump allocator.. you still have to allocate once per object, which can easily be an order of magnitude more work than what you have to do with an equivalent number of stack allocated objects. Your program doesn't magically know where those objects are, so it still also has to pay to stack allocate the pointers to keep track of the heap objects. Then you have the additional pointer-chasing slowing things down and the decreased effectiveness of the local CPU caches due to the additional level of indirection. A contiguous stack frame is a lot more likely to stay entirely within the CPU cache than a bunch of values scattered around the heap. Beyond that, and beyond the additional scanning you already mentioned, in the real world the heap is shared between all threads, which means there will be some amount of contention whenever you have to interact with the heap, although this is amortized some with TLABs (thread-local allocation buffers). You also have to consider the pointer rewriting that a generational GC will have to perform for the survivors of each generation, and that will not be tied strictly to function frames. The GC will run whenever it feels like it, so you may pay the cost of pointer rewriting for objects that are only used until the end of this function, just due to the coincidence of when the GC started working. I think (but could be wrong/outdated) that generational GCs almost all require both read and write barriers that perform some synchronization any time you interact with a heap allocated value, and this slows the program down even more compared to stack objects. (I believe that non-copying GCs don't need as many barriers, and that the barriers are only active during certain GC phases, which is beneficial to Go programs, but again, stack objects don't need any barriers at all, ever.) GCs are really cool, but stack allocated values are always better when they can be used. There's a reason that C# makes developer-defined value types available; they learned from the easily visible problems that Java has wrestled with caused by not allowing developers to define their own value types. Go took it a step further and got rid of developer-defined reference types altogether, so everything is a value type (arguably with the exception of the syntax sugar for the built-in map, slice, channel, and function pointer types), and even values allocated behind a pointer will still be stack allocated if escape analysis proves it won't cause problems.
socialdemocrat 5y ago
Java _needs_ lots of GC tuning parameters because you have practically no way of tuning the way your memory is used and organized in Java code. In Go you can actually do that. You can decide how data structures are nested, you can take pointers to the inside of a a block of memory. You could make e.g. a secondary allocator, allocating objects from a contiguous block of memory. Java doesn't allow those things, and thus it must instead give you lots of levers to pull on to tune the GC. It is just a different strategy of achieving the same thing: https://itnext.io/go-does-not-need-a-java-style-gc-ac99b8d26c60 https://itnext.io/go-does-not-need-a-java-style-gc-ac99b8d26...
- native_samples 5y ago
  That's the Go party line but not really true. Counter-example: The Go GC is tuned for HTTP servers at latency sensitive companies like Google. It therefore prioritizes latency over throughput to an astonishing degree, which means it is extremely bad at batch jobs - like compilers. What language is the Go compiler written in? Go. This isn't fixable by simply writing the code differently. What you're talking about is in the limit equivalent to not using a GCd language at all, and you can do that with Java too via the Unsafe allocators. But it's not a great idea to do that too much, because then you may as well just bite the bullet and write C++. Java doesn't actually need lots of GC tuning parameters. Actually most of the time you can ignore them, because the defaults balance latency and throughput for something reasonable for the vast majority of companies that aren't selling ad clicks. But, if you want, you can tell the JVM more about your app to get better results like whether it's latency or throughput sensitive. The parameters are there mostly to help people with unusual or obscure workloads where Go simply gives up and says "if you have this problem, Go is not for you".
  - delian66 5y ago
    > it is extremely bad at batch jobs - like compilers. > What language is the Go compiler written in? Go. I do not see what are you trying to say? The Go compiler is plenty fast in my experience, especially compared to say `javac`. The startup time of `javac` (and most java programs) is atrocious.
no_circuit 5y ago
Yes, my comments were targeted to Java and Scala. Java has paid the bills for me for many years. I'd use Java for just about anything except for high load infrastructure systems. And if you're in, or want to be in, that situation, then why risk finding out two years later that a GC-enabled app is suboptimal? I'd guess you'd have no choice if in order to hire developers, you had to choose a language that the people found fun to use.

apalmer 5y ago

no_circuit 5y ago
No worries. It is not meant to be quantitative. For a few years of my career that has been my experience. For this type of software, if I'm making the decision on what technology to use, it won't be any GC-based language. I'd rather not rely on promises that GC works great, or is very tunable. One could argue that I could just tune my services from time to time. But I'd just reduce the surface area for problems by not relying upon it at all -- both a technical and a business decision.
- throwawaylala1 5y ago
  If you're needing to fight the GC to prevent crashes or whatever then you have a system design issue not a tooling/language/ecosystem issue. There are exceptions to this but they're rare and not worth mentioning in a broad discussion like this. Sadly very few people take interest in learning how to design systems properly. Instead they find comfort in tools that allow them to over-engineer the problems away. Like falling into zealotry on things like FP, zero-overhead abstractions, "design patterns", containerization, manual memory management, etc, etc. These are all nice things when properly applied in context but they're not a substitute for making good system design decisions. Good system design starts with understanding what computers are good at and what they suck at. That's a lot more difficult than it sounds because today's abstractions try to hide what computers suck at. Example: Computers suck at networking. We have _a lot_ of complex layers to help make it feel somewhat reliable. But as a fundamental concept, it sucks. The day you network two computers together is the day you've opened yourself up to a world of hurt (think race conditions) - so, like, don't do it if you don't absolutely have to.
  - Karrot_Kream 5y ago

throwaway894345 5y ago

> The tricky thing is GC works most of the time, but if you are working at scale you really can't predict user behavior, and so all of those GC-tuning parameters that were set six months ago no longer work properly. A good portion of production outages are likely related to cascading failures due to too long GC pauses, and a good portion of developer time is spent testing and tuning GC parameters. It is easier to remove and/or just not allow GC languages at these levels in the first place. Getting rid of the GC doesn't absolve you of the problem, it just means that rather than tuning GC parameters, you've encoded usage assumptions in thousands of places scattered throughout your code base.

initplus 5y ago

It's impossible to spend any time tuning Go's GC parameters as they intentionally do not provide any. Go's GC is optimized for latency, it doesn't see the same kind of 1% peak latency issues you get in languages with a long tail of high latency pauses. Also consider API design - Java API (both in standard & third party libs) tend to be on the verbose side and build complex structures out of many nested objects. Most Go applications will have less nesting depth so it's inherently an easier GC problem. System designs that rely on allocating a huge amount of memory to a single process exist in a weird space - big enough that perf is really important, but small enough that single-process is still a viable design. Building massive monoliths that allocate hundreds of Gb's at peak load just doesn't seem "in vogue" anymore. If you are building a distributed system keeping any individual processes peak allocation to a reasonable size is almost automatic.

erik_seaberg 5y ago
You tune Go’s GC by rewriting your code. It’s like turning a knob but slower and riskier.
- coder543 5y ago
  You tune GC in Go by profiling allocations, CPU, and memory usage. Profiling shows you where the problems are, and Go has some surprisingly nice profiling tools built in. Unlike turning a knob, which has wide reaching and unpredictable effects that may cause problems to just move around from one part of your application to another, you can address the actual problems with near-surgical precision in Go. You can even add tests to the code to ensure that you're meeting the expected number of allocations along a certain code path if you need to guarantee against regressions... but the GC is so rarely the problem in Go compared to Java, it's just not something to worry about 99% of the time. If knobs had a "fix the problem" setting, they would already be set to that value. Instead, every value is a trade off, and since you have hundreds of knobs, you're playing an impossible optimization game with hundreds of parameters to try to find the set of parameter values that make your entire application perform the way you want it to. You might as well have a meta-tuner that just randomly turns the knobs to collect data on all the possible combinations of settings... and just hope that your next code change doesn't throw all that hard work out the window. Go gives you the tools to tune different parts of your code to behave in ways that are optimal for them. It's worth pointing out that languages like Rust and C++ also require you to tune allocations and deallocations... this is not strictly a GC problem. In those languages, like in Go, you have to address the actual problems instead of spinning knobs and hoping the problem goes away. The one time I have actually run up against Go's GC when writing code that was trying to push the absolute limits of what could be done on a fleet of rather resource constrained cloud instances, I wished I was writing Rust for this particular problem... I definitely wasn't wishing I could be spinning Java's GC knobs. But, I was still able to optimize things to work in Go the way I needed them to even in that case, even if the level of control isn't as granular as Rust would have provided.

EdwardDiego 5y ago

> A good portion of production outages are likely related to cascading failures due to too long GC pauses, and a good portion of developer time is spent testing and tuning GC parameters After 14 years in JVM dev in areas where latency and reliability are business critical, I disagree. Yes, excessive GC stop the world pauses can cause latency spikes, and excessive GC time is bad, and yes, when a new GC algorithm is released that you think might offer improvements, you test it thoroughly to determine if it's better or worse for your workload. But a "good portion" of outages and developer time? Nope. Most outages occur for the same old boring reasons - someone smashed the DB with an update that hits a pathological case and deadlocks processes using the same table, a DC caught fire, someone committed code with a very bad logical bug, someone considered a guru heard that gRPC was cool and used it without adequate code review and didn't understand that gRPC's load balancing defaults to pick first, etc. etc. The outages caused by GC were very very few. Outages caused by screw-ups or lack of understanding of subtleties of a piece of tech, as common as they are in every other field of development. Then there's the question of what outages GCed languages _don't_ suffer. I've never had to debug corrupted memory, or how a use after free bug let people exfiltrate data.

zinekeller 5y ago
> I've never had to debug corrupted memory You're lucky! When OpenJDK was still closed-sourced Hotspot from Sun, we have chased bugs that Sun confirmed was a defect on how Hotspot handle memory (and this is on a ECC'd system of course), although these days I can't remind of anything remotely related. > or how a use after free bug let people exfiltrate data. Technically you're just outsourcing it :)
- Karrot_Kream 5y ago
  > You're lucky! When OpenJDK was still closed-sourced Hotspot from Sun, we have chased bugs that Sun confirmed was a defect on how Hotspot handle memory (and this is on a ECC'd system of course), although these days I can't remind of anything remotely related. I mean sure. I remember having similar issues with early (< 2.3) Python builds as well. But in the last decade of my career, only a handful of outages were caused by Java GC issues. Most of them happened for a myriad of other architectural reasons.

exdsq 5y ago

I think I toggled with the GC for less than a week in my eight years experience including some systems stuff - maybe this is true at FANG scale but not for me!

eudoxus 5y ago

As many have replied, the available levers for 'GC-tuning' in go is almost non-existent. However, what we do have influence on is "GC Pressure" which is a very important metric we can move in the right direction if the application requires it.