4 ms·

One anecdote from working with .Net for over 20 years: I've had a few situations where someone (who isn't a programmer and/or doesn't work with .Net) insists th

One anecdote from working with .Net for over 20 years: I've had a few situations where someone (who isn't a programmer and/or doesn't work with .Net) insists that the application has a memory leak. First, I explain that garbage collected applications don't release memory immediately. Then I get sucked into a wild goose chase looking for a memory leak that doesn't exist. Finally, I point out that the behavior they see is normal, usually to some grumbling. From what I can tell, DATAS basically makes a .Net application have a normal memory footprint. Otherwise, .Net is quite a pig when it comes to memory. https://github.com/GWBasic/soft_matrix https://github.com/GWBasic/soft_matrix, implemented in Rust, generally has very low memory consumption. An earlier version that I wrote in C# would consume gigabytes of memory (and often run out of memory when run on Mono with the Bohem garbage collector.) --- > If startup perf is critical, DATAS is not for you This is one of my big frustrations with .net, (although I tend to look at how dependency injection is implemented as a bigger culprit.) It does make me wonder: How practical is it to just use traditional reference counting and then periodically do a mark-and-sweep? I know it's a very different approach than .net was designed for. (Because they deliberately decided that dereferencing an object should have no computational cost.) It's more of a rhetorical question.

nu11ptr 1y ago
> It does make me wonder: How practical is it to just use traditional reference counting and then periodically do a mark-and-sweep? I know it's a very different approach than .net was designed for. (Because they deliberately decided that dereferencing an object should have no computational cost.) It's more of a rhetorical question. This is what CPython does. The trade off is solidly worse allocator performance, however. You also have the reference counting overhead, which is not trivial unless it is deferred. There is always a connection between the allocator and collector. If you use a compacting collector (which I assumed .NET does), you get bump pointer allocation, which is very fast. However, if you use a non-compacting collector (mark-and-sweep is non-compacting), you would then fallback to a normal free list allocator (aka as "malloc") which has solidly higher overhead. You can see the impact of this (and reference counting) in any benchmark that builds a tree (and therefore is highly contended on allocation). This is also why languages that use free list allocation often have some sort of "arena" library, so they can have high speed bump pointer allocation in hot spots (and then free all that memory at once later on). BTW, reference counting, malloc/free performance also impact Rust, but given Rust's heavy reliance on the stack it often doesn't impact performance much (aka just doing less allocations). For allocation heavy code, many of us use MiMalloc one of the better malloc/free implementations.
- gwbas1c 1y ago
  So basically you're trading lowering RAM consumption for higher CPU consumption? FWIW: When I look at Azure costs, RAM tends to cost more than CPU. So the tradeoffs of using a "slower" memory manager might be justified.
  - nu11ptr 1y ago
    It depends on workload. It is difficult to quantify the trade offs without knowing that. The problem is in languages like C#/Java almost everything is an allocation, so I don't really think reference counting would work well there. I suspect this is the reason PyPy doesn't use reference counting, it is a big slowdown for CPython. Reference counting really only works well in languages with low allocations. Go mostly gets away with a non-compacting mark-sweep collector because it has low level control that allows many things to sit on the stack (like Rust/C/C++, etc.).
    - adgjlsfhk1 1y ago
      C# is a lot better than Java on this front since they support stack allocated structs
- whaleofatw2022 1y ago
  Dotnet does both mark and sweep as well as compaction, depends on what type of GC happens.
  - gwbas1c 1y ago
    In this case, we're discussing a case where mark-and-sweep is used to collect cyclic references, and it's implied that there are no generations. (Because otherwise, purely relying on reference counting means that cyclic references end up leaking unless things like weak references are used.) IE, the critical difference is that reference counting frees memory immediately; albeit at a higher CPU cost and needing to still perform a mark-and-sweep to clear out cyclic references.
- deleted 1y ago
neonsunset 1y ago
[dead]
SideburnsOfDoom 1y ago
> First, I explain that garbage collected applications don't release memory immediately. ... I point out that the behavior they see is normal yes, this is an easily overlooked point: Using memory when it going free is by design. It is often better to use use up cheap, unused memory instead of expensive CPU doing a GC. When memory is plentiful as it often is, then it is faster to just not run a GC yet. You're not in trouble unless you run short of memory, and a necessary GC does not free up enough. Then only can you call it an issue.
kg 1y ago
One of the main problems with refcounting is that unless your compiler/JIT are able to safely, aggressively optimize out reference increment/decrements, you can spend a ton of CPU time pointlessly bumping a counter up and down every time you enter a new function/method. This has been a problem for ObjC and Swift applications in the past AFAIK, though both of those compilers do a great job of optimizing that stuff out where possible. There are some other things that would probably be improvements coming along with refcounting though - you might be able to get rid of GC write barriers.
bob1029 1y ago
To be fair, there is an entire class of GC/memory problems that aren't technically a leak but manifest in effectively the same way. The most common one I see is LOH (Large Object Heap) fragmentation. When objects are promoted to the LOH the runtime doesnt bother with moving them around anymore. There is a way to explicitly compact the LOH but it can be a non-starter for a lot of applications. https://learn.microsoft.com/en-us/dotnet/api/system.runtime.gcsettings.largeobjectheapcompactionmode https://learn.microsoft.com/en-us/dotnet/api/system.runtime.... I've once exposed this as a button that a customer's IT department could click whenever they received an alert on memory utilization. The actual solution would have been to refactor the entire product to not pass gigantic blobs around all the time, but that wasn't in the cards for us.
WorldMaker 1y ago
> From what I can tell, DATAS basically makes a .Net application have a normal memory footprint. In Server environments. DATAS is an upgrade to garbage collection in "Server mode". Server GC assumed it could be the only thing running on a machine and could use as much memory as it wanted and so would just easily over-allocate memory much more than what it immediately needed. (As the article points out, it would start at a large fixed amount of memory times the number of CPU cores.) (As opposed to "Workstation GC" which has always tried to minimize memory consumption because it assumes it is running as only one of many apps on an end user system.) > (and often run out of memory when run on Mono with the Bohem garbage collector.) Not exactly a fair comparison between .NET's actual GC and Mono's old simpler GC before the merger. (Today's .NET shares the same GC on Windows and Linux [and macOS].) > This is one of my big frustrations with .net, (although I tend to look at how dependency injection is implemented as a bigger culprit.) Startup times have gotten a lot better in recent versions of .NET, AOT compiling has much improved (especially compared to the ancient ngen for anyone old enough to remember needing to use that for startup optimization), and while I agree .NET has seen a lot of terrible DI implementations the out-of-the-box one in Microsoft.Extensions does a lot of things right now, including avoiding a lot of Reflection in standard usage which was the big thing slowing down older DI systems. (I've seen people add Reflection based "helpers" back on top of the Microsoft.Extensions DI, but at that point that is a user problem more than a DI problem.) > It does make me wonder: How practical is it to just use traditional reference counting and then periodically do a mark-and-sweep? Technically the "mark" of "mark-and-sweep" can be implemented as traditional reference counting (and some of the earliest "mark-and-sweep" implementations did just that). It still only solves half the problem, though. Also, the optimizations made by modern "mark" systems come from that you don't need detailed counts, you just need tools equivalent to Bloom filters (what's the probability this is referenced at least once) and those can be much faster/more efficient to compute and use a lot less memory space than reference counters while doing that. If your concern is total memory consumption, traditional reference counting uses more space (if only just to store counts), and by itself doesn't solve fragmentation (the "sweep" part of "mark-and-sweep"). From a practical standpoint, combining "traditional reference counting" and a "mark-and-sweep" sounds to me like asking for a less efficient "mark-twice-and-sweep" algorithm.
- 1y ago

nu11ptr 1y ago

> It does make me wonder: How practical is it to just use traditional reference counting and then periodically do a mark-and-sweep? I know it's a very different approach than .net was designed for. (Because they deliberately decided that dereferencing an object should have no computational cost.) It's more of a rhetorical question. This is what CPython does. The trade off is solidly worse allocator performance, however. You also have the reference counting overhead, which is not trivial unless it is deferred. There is always a connection between the allocator and collector. If you use a compacting collector (which I assumed .NET does), you get bump pointer allocation, which is very fast. However, if you use a non-compacting collector (mark-and-sweep is non-compacting), you would then fallback to a normal free list allocator (aka as "malloc") which has solidly higher overhead. You can see the impact of this (and reference counting) in any benchmark that builds a tree (and therefore is highly contended on allocation). This is also why languages that use free list allocation often have some sort of "arena" library, so they can have high speed bump pointer allocation in hot spots (and then free all that memory at once later on). BTW, reference counting, malloc/free performance also impact Rust, but given Rust's heavy reliance on the stack it often doesn't impact performance much (aka just doing less allocations). For allocation heavy code, many of us use MiMalloc one of the better malloc/free implementations.

gwbas1c 1y ago
So basically you're trading lowering RAM consumption for higher CPU consumption? FWIW: When I look at Azure costs, RAM tends to cost more than CPU. So the tradeoffs of using a "slower" memory manager might be justified.
- nu11ptr 1y ago
  It depends on workload. It is difficult to quantify the trade offs without knowing that. The problem is in languages like C#/Java almost everything is an allocation, so I don't really think reference counting would work well there. I suspect this is the reason PyPy doesn't use reference counting, it is a big slowdown for CPython. Reference counting really only works well in languages with low allocations. Go mostly gets away with a non-compacting mark-sweep collector because it has low level control that allows many things to sit on the stack (like Rust/C/C++, etc.).
  - adgjlsfhk1 1y ago
    C# is a lot better than Java on this front since they support stack allocated structs
whaleofatw2022 1y ago
Dotnet does both mark and sweep as well as compaction, depends on what type of GC happens.
- gwbas1c 1y ago
  In this case, we're discussing a case where mark-and-sweep is used to collect cyclic references, and it's implied that there are no generations. (Because otherwise, purely relying on reference counting means that cyclic references end up leaking unless things like weak references are used.) IE, the critical difference is that reference counting frees memory immediately; albeit at a higher CPU cost and needing to still perform a mark-and-sweep to clear out cyclic references.
deleted 1y ago

neonsunset 1y ago

[dead]

SideburnsOfDoom 1y ago

> First, I explain that garbage collected applications don't release memory immediately. ... I point out that the behavior they see is normal yes, this is an easily overlooked point: Using memory when it going free is by design. It is often better to use use up cheap, unused memory instead of expensive CPU doing a GC. When memory is plentiful as it often is, then it is faster to just not run a GC yet. You're not in trouble unless you run short of memory, and a necessary GC does not free up enough. Then only can you call it an issue.

kg 1y ago

One of the main problems with refcounting is that unless your compiler/JIT are able to safely, aggressively optimize out reference increment/decrements, you can spend a ton of CPU time pointlessly bumping a counter up and down every time you enter a new function/method. This has been a problem for ObjC and Swift applications in the past AFAIK, though both of those compilers do a great job of optimizing that stuff out where possible. There are some other things that would probably be improvements coming along with refcounting though - you might be able to get rid of GC write barriers.

bob1029 1y ago

To be fair, there is an entire class of GC/memory problems that aren't technically a leak but manifest in effectively the same way. The most common one I see is LOH (Large Object Heap) fragmentation. When objects are promoted to the LOH the runtime doesnt bother with moving them around anymore. There is a way to explicitly compact the LOH but it can be a non-starter for a lot of applications. https://learn.microsoft.com/en-us/dotnet/api/system.runtime.gcsettings.largeobjectheapcompactionmode https://learn.microsoft.com/en-us/dotnet/api/system.runtime.... I've once exposed this as a button that a customer's IT department could click whenever they received an alert on memory utilization. The actual solution would have been to refactor the entire product to not pass gigantic blobs around all the time, but that wasn't in the cards for us.

WorldMaker 1y ago

> From what I can tell, DATAS basically makes a .Net application have a normal memory footprint. In Server environments. DATAS is an upgrade to garbage collection in "Server mode". Server GC assumed it could be the only thing running on a machine and could use as much memory as it wanted and so would just easily over-allocate memory much more than what it immediately needed. (As the article points out, it would start at a large fixed amount of memory times the number of CPU cores.) (As opposed to "Workstation GC" which has always tried to minimize memory consumption because it assumes it is running as only one of many apps on an end user system.) > (and often run out of memory when run on Mono with the Bohem garbage collector.) Not exactly a fair comparison between .NET's actual GC and Mono's old simpler GC before the merger. (Today's .NET shares the same GC on Windows and Linux [and macOS].) > This is one of my big frustrations with .net, (although I tend to look at how dependency injection is implemented as a bigger culprit.) Startup times have gotten a lot better in recent versions of .NET, AOT compiling has much improved (especially compared to the ancient ngen for anyone old enough to remember needing to use that for startup optimization), and while I agree .NET has seen a lot of terrible DI implementations the out-of-the-box one in Microsoft.Extensions does a lot of things right now, including avoiding a lot of Reflection in standard usage which was the big thing slowing down older DI systems. (I've seen people add Reflection based "helpers" back on top of the Microsoft.Extensions DI, but at that point that is a user problem more than a DI problem.) > It does make me wonder: How practical is it to just use traditional reference counting and then periodically do a mark-and-sweep? Technically the "mark" of "mark-and-sweep" can be implemented as traditional reference counting (and some of the earliest "mark-and-sweep" implementations did just that). It still only solves half the problem, though. Also, the optimizations made by modern "mark" systems come from that you don't need detailed counts, you just need tools equivalent to Bloom filters (what's the probability this is referenced at least once) and those can be much faster/more efficient to compute and use a lot less memory space than reference counters while doing that. If your concern is total memory consumption, traditional reference counting uses more space (if only just to store counts), and by itself doesn't solve fragmentation (the "sweep" part of "mark-and-sweep"). From a practical standpoint, combining "traditional reference counting" and a "mark-and-sweep" sounds to me like asking for a less efficient "mark-twice-and-sweep" algorithm.

1y ago