7 ms·

> Except in the special case ... IME it's the other way around, per-object individual lifetimes is a rare special case, in most real world code there will be m

> Except in the special case ... IME it's the other way around, per-object individual lifetimes is a rare special case, in most real world code there will be many related objects of the same or very similar lifetimes. In such code, tracking individual object lifetimes is overkill (in the end, memory management is all about lifetimes, and fewer individual lifetimes instead of many is always better because it means less work, both in manual and automatic memory management). Not having to think about object lifetimes is just very convenient, that's why GC languages were successful despite the significant under-the-hood complexity of a good garbage collector.

pron 3y ago
That's not at all a rare special case in most server applications. One way to see that is to consider how much of a program's working set is in threads' stacks vs the heap. If the most of the working set is in the heap, there's usually some non-trivial object lifetimes involved (i.e. cases where lifetimes can be encapsulated and abstracted away from client code). Yes, sometimes all of these can be taken care of by arenas (and there are languages, like Zig, that strive to be very arena-friendly), but that -- i.e. the case where all objects are easily arena-able -- is not the more common scenario.
- flohofwoe 3y ago
  Depends on the type of server I guess. I can easily imagine a situation where all allocations of a request are handled through a simple bump allocator, and once the request is done, the bump allocator is reset instead of 'freeing' each individual allocation.
  - cogman10 3y ago
    This would only work for fairly trivial applications. The moment you start adding things like http clients or databases you have to start considering having connection pools with lifetimes that don't strictly match a request lifecycle. Not saying such an application doesn't exist, it certainly does.
    - deleted 3y ago
      [deleted]
  - ncruces 3y ago
    That “simple bump allocator” is basically allocating everything on the thread's stack, as the GP mentioned. It's what you put on the heap that has complex lifetimes. Sometimes you can fix that with an arena. If you can't, you probably can't figure out when the last reference to it dies either.
    - deleted 3y ago
      [deleted]
  - jayd16 3y ago
    How did we go from "this is the most common case" to "I can imagine it?" Sure there are cases where a request can be handled by the stack but the point is that the more complex case is extremely common. Any kind of background/asynchronous work spawned from a request and your plans are shot.
    - thefaux 3y ago
      Yes and that is yet another case for encapsulation. For me, it is an antipattern for a short lived task to be directly creating long lived resources. Async work ideally is scheduled using message passing to the task manager which may have its own memory management strategy (heck, the background task may well be written in an entirely different language). I just feel that it is very common to have large portions of the application that fit the model of read input/write output, free all intermediate data upon completion. Due to a lack of encapsulation, however, we put unnecessary pressure on the allocator or garbage collector by mixing the short and long lived parts of our applications.
      - deleted 3y ago
        [deleted]
      - pron 3y ago
        > we put unnecessary pressure on the allocator or garbage collector by mixing the short and long lived parts of our applications. That is not how modern tracing GCs work, though. First, modern tracing GCs don't do any work to actively free any dead object (they don't even need to know about them). They work to compact the live ones. Second, modern tracing GCs automatically separate short-lived from long-lived objects and preserve them in different ways. While it's true that in principle a programmer could find a technique to optimally manage memory in a given program and that even a good GC would never be quite as optimal, in practice a GC would do a better job than you could unless you spend an inordinate amount of time figuring out the optimal strategy; furthermore, the optimal strategy has to contend with non-local effects, i.e. a change in one subroutine might have a global effect on what the optimal global strategy is. So as a first approximation, a good tracing GC will give you better memory management (performance-wise) per unit of programmer effort, and if you want even better performance -- give the GC more RAM. You're trading off RAM for less work on your part.
cratermoon 3y ago
In generational GCs there are two or more allocation regions. New objects are put in the "younger" generation, which is garbage collected separated from the other generations. This sort of resolves the issue of tracking individual object lifetimes by having all the short-lived objects subject to rapid GC. This means most of the effort tracking lifetimes is reserved for the fewer long-lived objects.
- lll-o-lll 3y ago
  It’s “medium term” objects that cause all the trouble. Async operations. That’s why .net invested heavily into zero-alloc tasks, as the gc would kill scaling in async heavy code.
zozbot234 3y ago
> IME it's the other way around, per-object individual lifetimes is a rare special case It depends on your application domain. But in most cases where objects have "individual lifetimes" you can still use reference counting, which has lower latency and memory overhead than tracing GC and interacts well with manual memory management. Tracing GC can then be "plugged in" for very specific cases, preferably using a high performance concurrent implementation much like https://github.com/chc4/samsara https://github.com/chc4/samsara (for Rust) or https://github.com/pebal/sgcl https://github.com/pebal/sgcl (for C++).
- pron 3y ago
  Reference counting does not have lower latency than a tracing GC nor is it more generally predictable. In terms of performance, it is usually worse than a modern tracing GC by most performance metrics. Its benefits are, indeed, lower memory footprint and that it can achieve reasonable latency even using a crude implementation. In other words, you'd reach for refcounting either if you mostly care about footprint or if you don't have a lot of resources to invest in implementing the GC. Modern concurrent tracing has very little impact on latency. First, compared to refcounting, the barriers are rarely invoked. Second, they compact the heap concurrently in the background, and like all mark-compact GCs, their amortized cost drops to zero as the RAM increases (the work is linear with the size of the working set, which is roughly constant for a given program, but it needs to be done less frequently the more RAM you have). That's why high-performance languages that rely heavily on a GC -- Java, C#, Go -- prefer tracing.
  - lll-o-lll 3y ago
    > nor is it more generally predictable Can you explain what you mean here? This does not match my experience or intuition.
    - pron 3y ago
      In theory, refcounting and tracing can behave similarly [1], but assuming we're speaking about their implementations in the industry (rather elaborate tracing GCs; rather crude refcounting GCs) then a refcounting GC would do some work as soon the refcount for a particular object drops to zero. When exactly that happens and how much work there is to do (and sometimes, what the fragmentation effect on heap is) are local properties that are not very predictable. In contrast, the amount of work a tracing GC needs to do when compacting is directly proportional to the program's working set and the frequency in which it needs to do that work is proportional to the allocation rate -- both of which are fairly regular and predictable global properties for a given program. For stop-the-world GCs there was then the matter of the highly unpredictable exact timing of a large STW pause, but modern concurrent GCs don't collect anything in STW pauses anymore. They only have very short (sub 1ms) and constant-time pauses and no surprise throttling as long as the allocation rate is within an acceptable range. So all in all, you pay a fairly fixed tax on the CPU (and if you want to pay less, just add more RAM) and virtually no impact on latency. [1]: https://www.cs.cornell.edu/courses/cs6120/2019fa/blog/unified-theory-gc/ https://www.cs.cornell.edu/courses/cs6120/2019fa/blog/unifie...

pron 3y ago

That's not at all a rare special case in most server applications. One way to see that is to consider how much of a program's working set is in threads' stacks vs the heap. If the most of the working set is in the heap, there's usually some non-trivial object lifetimes involved (i.e. cases where lifetimes can be encapsulated and abstracted away from client code). Yes, sometimes all of these can be taken care of by arenas (and there are languages, like Zig, that strive to be very arena-friendly), but that -- i.e. the case where all objects are easily arena-able -- is not the more common scenario.

flohofwoe 3y ago
Depends on the type of server I guess. I can easily imagine a situation where all allocations of a request are handled through a simple bump allocator, and once the request is done, the bump allocator is reset instead of 'freeing' each individual allocation.
- cogman10 3y ago
  This would only work for fairly trivial applications. The moment you start adding things like http clients or databases you have to start considering having connection pools with lifetimes that don't strictly match a request lifecycle. Not saying such an application doesn't exist, it certainly does.
  - deleted 3y ago
    [deleted]
- ncruces 3y ago
  That “simple bump allocator” is basically allocating everything on the thread's stack, as the GP mentioned. It's what you put on the heap that has complex lifetimes. Sometimes you can fix that with an arena. If you can't, you probably can't figure out when the last reference to it dies either.
  - deleted 3y ago
    [deleted]
- jayd16 3y ago
  How did we go from "this is the most common case" to "I can imagine it?" Sure there are cases where a request can be handled by the stack but the point is that the more complex case is extremely common. Any kind of background/asynchronous work spawned from a request and your plans are shot.
  - thefaux 3y ago
    Yes and that is yet another case for encapsulation. For me, it is an antipattern for a short lived task to be directly creating long lived resources. Async work ideally is scheduled using message passing to the task manager which may have its own memory management strategy (heck, the background task may well be written in an entirely different language). I just feel that it is very common to have large portions of the application that fit the model of read input/write output, free all intermediate data upon completion. Due to a lack of encapsulation, however, we put unnecessary pressure on the allocator or garbage collector by mixing the short and long lived parts of our applications.
    - deleted 3y ago
      [deleted]
    - pron 3y ago
      > we put unnecessary pressure on the allocator or garbage collector by mixing the short and long lived parts of our applications. That is not how modern tracing GCs work, though. First, modern tracing GCs don't do any work to actively free any dead object (they don't even need to know about them). They work to compact the live ones. Second, modern tracing GCs automatically separate short-lived from long-lived objects and preserve them in different ways. While it's true that in principle a programmer could find a technique to optimally manage memory in a given program and that even a good GC would never be quite as optimal, in practice a GC would do a better job than you could unless you spend an inordinate amount of time figuring out the optimal strategy; furthermore, the optimal strategy has to contend with non-local effects, i.e. a change in one subroutine might have a global effect on what the optimal global strategy is. So as a first approximation, a good tracing GC will give you better memory management (performance-wise) per unit of programmer effort, and if you want even better performance -- give the GC more RAM. You're trading off RAM for less work on your part.

cratermoon 3y ago

In generational GCs there are two or more allocation regions. New objects are put in the "younger" generation, which is garbage collected separated from the other generations. This sort of resolves the issue of tracking individual object lifetimes by having all the short-lived objects subject to rapid GC. This means most of the effort tracking lifetimes is reserved for the fewer long-lived objects.

lll-o-lll 3y ago
It’s “medium term” objects that cause all the trouble. Async operations. That’s why .net invested heavily into zero-alloc tasks, as the gc would kill scaling in async heavy code.

zozbot234 3y ago

> IME it's the other way around, per-object individual lifetimes is a rare special case It depends on your application domain. But in most cases where objects have "individual lifetimes" you can still use reference counting, which has lower latency and memory overhead than tracing GC and interacts well with manual memory management. Tracing GC can then be "plugged in" for very specific cases, preferably using a high performance concurrent implementation much like https://github.com/chc4/samsara https://github.com/chc4/samsara (for Rust) or https://github.com/pebal/sgcl https://github.com/pebal/sgcl (for C++).

pron 3y ago
Reference counting does not have lower latency than a tracing GC nor is it more generally predictable. In terms of performance, it is usually worse than a modern tracing GC by most performance metrics. Its benefits are, indeed, lower memory footprint and that it can achieve reasonable latency even using a crude implementation. In other words, you'd reach for refcounting either if you mostly care about footprint or if you don't have a lot of resources to invest in implementing the GC. Modern concurrent tracing has very little impact on latency. First, compared to refcounting, the barriers are rarely invoked. Second, they compact the heap concurrently in the background, and like all mark-compact GCs, their amortized cost drops to zero as the RAM increases (the work is linear with the size of the working set, which is roughly constant for a given program, but it needs to be done less frequently the more RAM you have). That's why high-performance languages that rely heavily on a GC -- Java, C#, Go -- prefer tracing.
- lll-o-lll 3y ago
  > nor is it more generally predictable Can you explain what you mean here? This does not match my experience or intuition.
  - pron 3y ago
    In theory, refcounting and tracing can behave similarly [1], but assuming we're speaking about their implementations in the industry (rather elaborate tracing GCs; rather crude refcounting GCs) then a refcounting GC would do some work as soon the refcount for a particular object drops to zero. When exactly that happens and how much work there is to do (and sometimes, what the fragmentation effect on heap is) are local properties that are not very predictable. In contrast, the amount of work a tracing GC needs to do when compacting is directly proportional to the program's working set and the frequency in which it needs to do that work is proportional to the allocation rate -- both of which are fairly regular and predictable global properties for a given program. For stop-the-world GCs there was then the matter of the highly unpredictable exact timing of a large STW pause, but modern concurrent GCs don't collect anything in STW pauses anymore. They only have very short (sub 1ms) and constant-time pauses and no surprise throttling as long as the allocation rate is within an acceptable range. So all in all, you pay a fairly fixed tax on the CPU (and if you want to pay less, just add more RAM) and virtually no impact on latency. [1]: https://www.cs.cornell.edu/courses/cs6120/2019fa/blog/unified-theory-gc/ https://www.cs.cornell.edu/courses/cs6120/2019fa/blog/unifie...