12 ms·

GC Tuning Confessions of a Performance Engineer

35 points by luu 11y ago

phamilton 11y ago
For some reason, GC is on of those areas that I have zero desire to learn sufficiently to tune. That's probably why OP has a successful career as a consultant. When making tradeoffs in development, a simpler GC with predictable behavior is worth losing a lot of raw performance to me. Thus I find myself drawn to platforms like erlang/elixir or Haskell where GC is isolated. I suppose the rational approach is build on Hotspot and when it becomes a problem hire a consultant for instant performance gains (assuming low hanging fruit is in abundance).
- pron 11y ago
  In 80% of the cases, simple GC ergonomics (i.e. let HotSpot figure out the tuning) are more than ok, and in 95% of the cases, trivial tuning is enough. The thing with Erlang is that the GC doesn't work on any shared memory data structure (like ETS), so you pretty much have to delegate any shared data to an out-of-process database, even in simple cases that are easily addressed with ConcurrentHashMap, ConcurrentSkipListMap/Set, or CopyOnWriteArrayList/Set on the JVM.
  - phamilton 11y ago
    I generally use an Agent in Elixir for anything shared and there's very little cognitive overhead (though perf does take a hit). Agent.cast/2 is helpful for low latency modifications (at the cost of losing back pressure). As stated, it's a tradeoff.
    - pron 11y ago
      But, if I'm not mistaken, an agent isn't concurrent. You make your shared data a bottleneck which is exactly what concurrent data strctcures are meant to prevent.
  - duaneb 11y ago
    Shared memory data structures are rarely necessary. When one is, I would be willing to go the extra mile to facilitate it. Seems like a reasonable tradeoff.
  - rdtsc 11y ago
    Good point, but ETS is still managed and owned by a process. So it still can be cleaned up taken care of. Moreover at the API level ETS even behaves as if there is a process that manages data (even though implementation-wise it is all done in C). Also in Erlang large binaries are not copied but shared as well. And whether large shared data-structures are needed is dependent on the application. I imagine since you've done work with spatial relationships, in that case for anything interesting to happen data has to be indexed, shared, queried in one place. But say building a large concurrent chat or messaging application with many but small state machine actor that might have lots of peer to peer interaction, then shared data is not as crucial.
- rdtsc 11y ago
  Yeah that is why I like the BEAM VM as well. It is a really awesome piece of engineering build for massive concurrency and fault tolerance. It is nice to see the whole Elixir ecosystem take advantage of it. That model of concurrency is important to reduce cognitive load on the programmer. Having concurrent units and mutable state isolated makes it much easier to understand what is going on, rather than having some large class or shared memory state modified from many code paths concurrently (maybe via callbacks, threads, signals, etc.)
jeffcoat 11y ago
The slides are interesting enough that I went looking to see if the actual talk that goes with them was online anywhere; found it at http://chariotsolutions.com/screencast/philly-ete-2015-7-monica-beckwith-gc-tuning-confessions-of-a-performance-engineer/ http://chariotsolutions.com/screencast/philly-ete-2015-7-mon...
banachtarski 11y ago
Having worked with all sorts of GCs in the past, I basically stopped using them altogether some years ago in favor of manual memory management. I have to say, it's been liberating and so much easier to write performant code since I know what memory I need instead of relying on the computer to guess (and giving the computer hints as to how to guess). I have no doubt that GC research and systems is making good forward progress, but I can't imagine myself using them again.
- pron 11y ago
  I love GCs. They give you better memory throughput (in exchange for more footprint and higher latency, although latency can be made rather low), and they let you build and use very scalable concurrent data structures. On large machines with lots of cores and lots of RAM, they let you work with large, in-memory data sets very efficiently.
  - duaneb 11y ago
    As with everything in engineering, this is a tradeoff. Not all access patterns work well with mark-and-sweep (or equivalent full traversal patterns). In particular, without needing to release or allocate memory, a GC pass wastes cycles. Furthermore, data structures must be compatible with the GC: traversal is proportional to the number of pointers (rooted or dangling) in the heap and stack at the time of running. Generational garbage collection helps this, but for large numbers of pointers, you're going to see performance decrease unless at some point you move the pointers out of collection contention--think an allocation pool to reduce the depth of traversal early. I believe google just released a library to help with this, though the semantics are specific enough you could easily make your problems much worse. GC is never a flat win.
    - pron 11y ago
      > traversal is proportional to the number of pointers (rooted or dangling) in the heap and stack at the time of running. That's not quite how it works. Traversal is proportional to the number of pointers changed since the last collection (HotSpot's GCs do card marking). I do agree there are tradeoffs, but they're much more nuanced than that. There is certainly a footprint tradeoff, and there is a latency tradeoff (that can be mitigated).
decafbad 11y ago
presentation video: https://www.youtube.com/watch?v=PnPzeavMjmE https://www.youtube.com/watch?v=PnPzeavMjmE
nickpsecurity 11y ago
Interesting article. What I see is another failure of industry to solve the root of the problem: different GC strategies work for different applications and should be selectable. Might even allow pluggable GC's. This is the path taken in JX Operating System and certain academic works on improving Java runtime. JX's first tier of memory management just gives a certain amount of resources a component/process/app can use. The second tier is its GC, which can be different per application. So, you could partition your application into components with associated GC's that focused on throughput, low-latency, or productivity. Then the problem shifts to making an optimal scheduler for the app and GC runtimes. There's already good schedulers, strategies for building them, and even tools for automated planning of scheduling strategies. Any of these might be integrated to optimize that aspect. Overall, using pluggable garbage collectors, feeding a planner some constraints, and compiling the result sounds much easier that this person's day job. Embedded Java already does some of this while some systems did it in hardware to avoid most issues entirely. Enterprise Java should adopt such a method. Meanwhile, reading of these nightmares, I'll continue avoiding such GC-based tools wherever possible in my work. JX Operating System for reference http://www4.cs.fau.de/Projects/JX/publications/jx-usenix.pdf http://www4.cs.fau.de/Projects/JX/publications/jx-usenix.pdf
jokoon 11y ago
I wonder, isn't it working on the stack much faster than working on the heap anyway? So when you query data on the heap, shouldn't you just query larger objects to put it on the stack, and work from there, instead of using the heap so often ? With that in mind wouldn't that render garbage collecting almost irrelevant if your code is well designed, by not working too much on the heap ? It's true that more ram makes the GC more relevant, but if it's an excuse for negligent software design, maybe GCs are not such a good idea. It's good to have features that make the job of the programmer easier, but if it only save the time of the skilled programmers who knows a little about how it works underneath, is it such a good idea ? You can hardly convince that simple tools and predictable behaviors in machines are not the safest way of having fair results. A programming language is already a big shortcut to work faster, and I doubt you should try to work even faster if it means creating new drawbacks.
- outworlder 11y ago
  > I wonder, isn't it working on the stack much faster than working on the heap anyway? Working on GC-managed memory is faster still. Remember, not all allocations involve an actual heap allocation. Conversely, not all deletions cause a 'free()' operation on the heap.
  - vitalyd 11y ago
    > Working on GC-managed memory is faster still. Care to elaborate?
    - dumael 11y ago
      Replying to GP comment and you as such. Hypothetically, (I'm not sure how widespread this technique is), some (Java) compilers turn heap allocations into stack allocations when they can prove that transitively an allocated object does not become reachable from other threads. This general technique "escape analysis", i.e. "a reference to this object /escapes/ a given reachability type[1]", can be used to transform general heap allocations into thread local or stack allocations. [1] Let me define reachability type as {can be accessed from anything, can be accessed from this thread only but is of indefinite lifetime, can be accessed from this thread only and it's reference is not take by any object allocated onto my local heap, ...}
jimrandomh 11y ago
Performance is nice, but correctness is more important. Complex programs written without any GC tend to have use-after-free and double-free vulnerabilities. Programs written using automatic reference counting only (without a way to catch cycles) tend to have memory leaks. If GC is killing performance, then that is a failure of the specific GC or programming language design. Don't blame the applications programmers for wanting GC semantics; that's a completely reasonable thing to want and something that language designers are capable of providing. In the specific case of Java, I think the real problem is not having value objects (like C#'s struct) or generics with non-reference types. Serious GC problems happen not because GC is too hard, but because in Java an ArrayList<int> of a million items makes a million tiny objects. This problem is entirely Java-specific.
- pjmlp 11y ago
  > This problem is entirely Java-specific. Fully correct. Back when Java was still in its infancy, we had Oberon(-2), Component Pascal and Modula-3 as system programming languages that allowed for C like memory allocation, coupled with a GC. But Java was the one that made GCs finally go mainstream and now many equate GC with Java ones[0], thinking all GC are made alike. [0] Forgetting in the process that Hotspot is just one JVM among many.
- astral303 11y ago
  Specifically about ArrayList<Integer> -- there are specifically a bunch of projects (Goldman Sachs Collections, trove, FastUtil, Koloboke) that solve the array of primitives problem. I would say that's a Java-standard-libary-specific problem.
  - TheLoneWolfling 11y ago
    And then you have to do a copy just to pass it around to any other external libraries. And you lose the advantages of generics. You end up with 9 copies of everything, that are 99% the same except for a couple find-replaces. If not more. (For instance, if you have a method that takes two generic arrays, you need 81 copies! Even if they are the same type you still need 36 (!) copies.) Not a good solution.
TheLoneWolfling 11y ago
I don't see why people put so much work into GC before even bothering to do any work to avoid objects becoming garbage in the first place beyond simple escape analysis.

35 points by luu 11y ago

For some reason, GC is on of those areas that I have zero desire to learn sufficiently to tune. That's probably why OP has a successful career as a consultant. When making tradeoffs in development, a simpler GC with predictable behavior is worth losing a lot of raw performance to me. Thus I find myself drawn to platforms like erlang/elixir or Haskell where GC is isolated. I suppose the rational approach is build on Hotspot and when it becomes a problem hire a consultant for instant performance gains (assuming low hanging fruit is in abundance).

pron 11y ago
In 80% of the cases, simple GC ergonomics (i.e. let HotSpot figure out the tuning) are more than ok, and in 95% of the cases, trivial tuning is enough. The thing with Erlang is that the GC doesn't work on any shared memory data structure (like ETS), so you pretty much have to delegate any shared data to an out-of-process database, even in simple cases that are easily addressed with ConcurrentHashMap, ConcurrentSkipListMap/Set, or CopyOnWriteArrayList/Set on the JVM.
- phamilton 11y ago
  I generally use an Agent in Elixir for anything shared and there's very little cognitive overhead (though perf does take a hit). Agent.cast/2 is helpful for low latency modifications (at the cost of losing back pressure). As stated, it's a tradeoff.
  - pron 11y ago
    But, if I'm not mistaken, an agent isn't concurrent. You make your shared data a bottleneck which is exactly what concurrent data strctcures are meant to prevent.
- duaneb 11y ago
  Shared memory data structures are rarely necessary. When one is, I would be willing to go the extra mile to facilitate it. Seems like a reasonable tradeoff.
- rdtsc 11y ago
  Good point, but ETS is still managed and owned by a process. So it still can be cleaned up taken care of. Moreover at the API level ETS even behaves as if there is a process that manages data (even though implementation-wise it is all done in C). Also in Erlang large binaries are not copied but shared as well. And whether large shared data-structures are needed is dependent on the application. I imagine since you've done work with spatial relationships, in that case for anything interesting to happen data has to be indexed, shared, queried in one place. But say building a large concurrent chat or messaging application with many but small state machine actor that might have lots of peer to peer interaction, then shared data is not as crucial.
rdtsc 11y ago
Yeah that is why I like the BEAM VM as well. It is a really awesome piece of engineering build for massive concurrency and fault tolerance. It is nice to see the whole Elixir ecosystem take advantage of it. That model of concurrency is important to reduce cognitive load on the programmer. Having concurrent units and mutable state isolated makes it much easier to understand what is going on, rather than having some large class or shared memory state modified from many code paths concurrently (maybe via callbacks, threads, signals, etc.)

jeffcoat 11y ago

The slides are interesting enough that I went looking to see if the actual talk that goes with them was online anywhere; found it at http://chariotsolutions.com/screencast/philly-ete-2015-7-monica-beckwith-gc-tuning-confessions-of-a-performance-engineer/ http://chariotsolutions.com/screencast/philly-ete-2015-7-mon...

banachtarski 11y ago

Having worked with all sorts of GCs in the past, I basically stopped using them altogether some years ago in favor of manual memory management. I have to say, it's been liberating and so much easier to write performant code since I know what memory I need instead of relying on the computer to guess (and giving the computer hints as to how to guess). I have no doubt that GC research and systems is making good forward progress, but I can't imagine myself using them again.

pron 11y ago
I love GCs. They give you better memory throughput (in exchange for more footprint and higher latency, although latency can be made rather low), and they let you build and use very scalable concurrent data structures. On large machines with lots of cores and lots of RAM, they let you work with large, in-memory data sets very efficiently.
- duaneb 11y ago
  As with everything in engineering, this is a tradeoff. Not all access patterns work well with mark-and-sweep (or equivalent full traversal patterns). In particular, without needing to release or allocate memory, a GC pass wastes cycles. Furthermore, data structures must be compatible with the GC: traversal is proportional to the number of pointers (rooted or dangling) in the heap and stack at the time of running. Generational garbage collection helps this, but for large numbers of pointers, you're going to see performance decrease unless at some point you move the pointers out of collection contention--think an allocation pool to reduce the depth of traversal early. I believe google just released a library to help with this, though the semantics are specific enough you could easily make your problems much worse. GC is never a flat win.
  - pron 11y ago
    > traversal is proportional to the number of pointers (rooted or dangling) in the heap and stack at the time of running. That's not quite how it works. Traversal is proportional to the number of pointers changed since the last collection (HotSpot's GCs do card marking). I do agree there are tradeoffs, but they're much more nuanced than that. There is certainly a footprint tradeoff, and there is a latency tradeoff (that can be mitigated).

decafbad 11y ago

presentation video: https://www.youtube.com/watch?v=PnPzeavMjmE https://www.youtube.com/watch?v=PnPzeavMjmE

nickpsecurity 11y ago

Interesting article. What I see is another failure of industry to solve the root of the problem: different GC strategies work for different applications and should be selectable. Might even allow pluggable GC's. This is the path taken in JX Operating System and certain academic works on improving Java runtime. JX's first tier of memory management just gives a certain amount of resources a component/process/app can use. The second tier is its GC, which can be different per application. So, you could partition your application into components with associated GC's that focused on throughput, low-latency, or productivity. Then the problem shifts to making an optimal scheduler for the app and GC runtimes. There's already good schedulers, strategies for building them, and even tools for automated planning of scheduling strategies. Any of these might be integrated to optimize that aspect. Overall, using pluggable garbage collectors, feeding a planner some constraints, and compiling the result sounds much easier that this person's day job. Embedded Java already does some of this while some systems did it in hardware to avoid most issues entirely. Enterprise Java should adopt such a method. Meanwhile, reading of these nightmares, I'll continue avoiding such GC-based tools wherever possible in my work. JX Operating System for reference http://www4.cs.fau.de/Projects/JX/publications/jx-usenix.pdf http://www4.cs.fau.de/Projects/JX/publications/jx-usenix.pdf

jokoon 11y ago

I wonder, isn't it working on the stack much faster than working on the heap anyway? So when you query data on the heap, shouldn't you just query larger objects to put it on the stack, and work from there, instead of using the heap so often ? With that in mind wouldn't that render garbage collecting almost irrelevant if your code is well designed, by not working too much on the heap ? It's true that more ram makes the GC more relevant, but if it's an excuse for negligent software design, maybe GCs are not such a good idea. It's good to have features that make the job of the programmer easier, but if it only save the time of the skilled programmers who knows a little about how it works underneath, is it such a good idea ? You can hardly convince that simple tools and predictable behaviors in machines are not the safest way of having fair results. A programming language is already a big shortcut to work faster, and I doubt you should try to work even faster if it means creating new drawbacks.

outworlder 11y ago
> I wonder, isn't it working on the stack much faster than working on the heap anyway? Working on GC-managed memory is faster still. Remember, not all allocations involve an actual heap allocation. Conversely, not all deletions cause a 'free()' operation on the heap.
- vitalyd 11y ago
  > Working on GC-managed memory is faster still. Care to elaborate?
  - dumael 11y ago
    Replying to GP comment and you as such. Hypothetically, (I'm not sure how widespread this technique is), some (Java) compilers turn heap allocations into stack allocations when they can prove that transitively an allocated object does not become reachable from other threads. This general technique "escape analysis", i.e. "a reference to this object /escapes/ a given reachability type[1]", can be used to transform general heap allocations into thread local or stack allocations. [1] Let me define reachability type as {can be accessed from anything, can be accessed from this thread only but is of indefinite lifetime, can be accessed from this thread only and it's reference is not take by any object allocated onto my local heap, ...}

jimrandomh 11y ago

Performance is nice, but correctness is more important. Complex programs written without any GC tend to have use-after-free and double-free vulnerabilities. Programs written using automatic reference counting only (without a way to catch cycles) tend to have memory leaks. If GC is killing performance, then that is a failure of the specific GC or programming language design. Don't blame the applications programmers for wanting GC semantics; that's a completely reasonable thing to want and something that language designers are capable of providing. In the specific case of Java, I think the real problem is not having value objects (like C#'s struct) or generics with non-reference types. Serious GC problems happen not because GC is too hard, but because in Java an ArrayList<int> of a million items makes a million tiny objects. This problem is entirely Java-specific.

pjmlp 11y ago
> This problem is entirely Java-specific. Fully correct. Back when Java was still in its infancy, we had Oberon(-2), Component Pascal and Modula-3 as system programming languages that allowed for C like memory allocation, coupled with a GC. But Java was the one that made GCs finally go mainstream and now many equate GC with Java ones[0], thinking all GC are made alike. [0] Forgetting in the process that Hotspot is just one JVM among many.
astral303 11y ago
Specifically about ArrayList<Integer> -- there are specifically a bunch of projects (Goldman Sachs Collections, trove, FastUtil, Koloboke) that solve the array of primitives problem. I would say that's a Java-standard-libary-specific problem.
- TheLoneWolfling 11y ago
  And then you have to do a copy just to pass it around to any other external libraries. And you lose the advantages of generics. You end up with 9 copies of everything, that are 99% the same except for a couple find-replaces. If not more. (For instance, if you have a method that takes two generic arrays, you need 81 copies! Even if they are the same type you still need 36 (!) copies.) Not a good solution.

TheLoneWolfling 11y ago

I don't see why people put so much work into GC before even bothering to do any work to avoid objects becoming garbage in the first place beyond simple escape analysis.