4 ms·

> Arc is the standard solution, but, for multi-threaded code, it can be expensive (though also more predictable, in fairness) than having a GC. Do you have mea

> Arc is the standard solution, but, for multi-threaded code, it can be expensive (though also more predictable, in fairness) than having a GC. Do you have measurements to back that up? I would expect it to be a lot cheaper than GC because GCed language such as Java/Go/etc. GC every heap allocation (and Java in particular doesn't have unboxed objects, so there are a lot of heap allocations!) where Arc<RequestState> would bundle a bunch of things together. That said, I'm not in love with the Arc solution either. It's not just the performance impact. The programming model of scoped concurrency is just much more pleasant. Giving up that and then also having to deal with putting everything into an Arc, and likely also then wrapping it in a MutexLock, even when it should really be one thread at a time dealing with the value in question, just doesn't do anything good for readability and writability of code...

charcircuit 3y ago
>because GCed language such as Java/Go/etc. GC every heap allocation GC is not run on every allocation.
- o11c 3y ago
  They don't run GC, but they subject the extra pointers to GC, whereas good RC languages generally support values. Doing nothing is usually cheaper than doing something. This is more a Java problem than a GC problem; C# supports values just fine, and some heroic JVM-targetting languages also manage it.
  - manwe150 3y ago
    But then mark and sweep also doesn’t have to free the memory (it just does a bulk sweep occasionally), and RC languages do need to free memory used to hold the values later (they have to run expensive compaction and locking steps). So definitely more a Java problem with poor allocation hoisting and reuse than a difference between the RC and mark and sweep implementations of GC.
    - scottlamb 3y ago
      Rust isn't an "RC" language like Java is a "GC" language. It's more flexible than that. Rust has actual unboxed/value structs; it has Box (heap allocation without reference counting); it has Rc (single-thread/no-atomic reference counting); it has Arc (atomic reference counting); it has epoch gc libraries. And, regarding this phrase: > RC languages do need to free memory used to hold the values later It also has arena allocator libraries, so you can group allocations that will be freed at a similar time. You don't need to walk through these to find them all and individually free them. Any Drop impls (destructors/finalizers) get handled via a lifo; then the arena's relatively few, large blocks get freed. I've used this in C++ to squeeze some more performance out of a high-request-rate server. I haven't done it in Rust, but it's the same idea (but safer of course). btw, little confused by this: > [RC languages?] have to run expensive compaction and locking steps Was this meant to refer to GC languages? I've never seen a non-GC language that does compaction (we're talking about the same thing? moving all the heap allocations around to reduce fragmentation?) That's more of a GC language thing that e.g. Java does; it would require the language/runtime to be able to precisely spot pointers and be able to throw in a trap when one it's changing is dereferenced or to entirely stop the world first, and I don't know why you'd take on that complexity and expense if you're not doing GC...
      - manwe150 3y ago
        Yes, rust is a great language. I love what it does. It also has memory leaks, because it is missing the cleanup part of the GC so it can’t deal with reference counter cycles. Well, there is boehm if you do want to take on that complexity. But no, I am referring to the book keeping operations that are done internal to the malloc and free calls. They are largely hidden from you as the user of the API, but you still have to pay for them in runtime. I thought rust didn’t have any safe arena allocators, but that some people make them by tricking the system with indexes instead and such, though word was they might come soon https://news.ycombinator.com/item?id=33403324 https://news.ycombinator.com/item?id=33403324
- scottlamb 3y ago
  Are you interpreting "GC every heap allocation" as "each heap allocation causes an entire cycle of GC"? That's...not what that phrase means.
  - charcircuit 3y ago
    Yes I am. Java's GC can stop the world at any safepoint. It isn't limited to when objects are allocated and it doesn't do it every time.
    - scottlamb 3y ago
      I'm aware. A cycle per allocation would be absurdly expensive. "GC every heap allocation" actually means that every heap allocation is subject to garbage collection and has to be traversed on every cycle in which it's reachable. If the GC algorithm uses compaction and/or generations, it gets moved around too. That all has a cost.
yencabulator 3y ago
The reason Arc hurts is that it's global and atomic. Multiple cores competing to inc/dec a shared reference count is pretty much a worst case scenario for modern CPUs. Under modern garbage collection, there's very little need to coordinate across cores, just some barriers every now and then to mark things safe.
- scottlamb 3y ago
  True, but impact depends on both total core count you're using in the process and what you're doing with Arc. I'm imagining Arc<RequestState> with low concurrency on each RequestState so impact of cache line bouncing should be negligible. If you're talking about Arc<GlobalStuff>, I'd probably use Box::leak instead. There's also a few crates that do epoch gc. I haven't really written super high request rate multicore stuff in Rust, but I have in C++, and there for global-ish occasionally updated config stuff, I used a epoch gc implementation on top of Linux rseq to totally eliminate the effect you're describing.
  - yencabulator 3y ago
    The classic case where Arc hurts and GC really shines is Arc<Config>. Arc makes otherwise read-only activity write.
    - pkolaczk 3y ago
      How many independent things do you need to access the config? A dozen? Then there is a dozen calls to inc the RC at startup, but there is no later cost to read from object behind Arc. So you may read your config a million times and Arc changes nothing. While GC will have to scan the Config structure every time a full GC is needed, so there is an added cost.
      - yencabulator 3y ago
        In the trouble scenario, config may change at any time. A getter function has to return a freshly-incremented Arc<Config>, which lives for roughly one request.

charcircuit 3y ago

>because GCed language such as Java/Go/etc. GC every heap allocation GC is not run on every allocation.

o11c 3y ago
They don't run GC, but they subject the extra pointers to GC, whereas good RC languages generally support values. Doing nothing is usually cheaper than doing something. This is more a Java problem than a GC problem; C# supports values just fine, and some heroic JVM-targetting languages also manage it.
- manwe150 3y ago
  But then mark and sweep also doesn’t have to free the memory (it just does a bulk sweep occasionally), and RC languages do need to free memory used to hold the values later (they have to run expensive compaction and locking steps). So definitely more a Java problem with poor allocation hoisting and reuse than a difference between the RC and mark and sweep implementations of GC.
  - scottlamb 3y ago
    Rust isn't an "RC" language like Java is a "GC" language. It's more flexible than that. Rust has actual unboxed/value structs; it has Box (heap allocation without reference counting); it has Rc (single-thread/no-atomic reference counting); it has Arc (atomic reference counting); it has epoch gc libraries. And, regarding this phrase: > RC languages do need to free memory used to hold the values later It also has arena allocator libraries, so you can group allocations that will be freed at a similar time. You don't need to walk through these to find them all and individually free them. Any Drop impls (destructors/finalizers) get handled via a lifo; then the arena's relatively few, large blocks get freed. I've used this in C++ to squeeze some more performance out of a high-request-rate server. I haven't done it in Rust, but it's the same idea (but safer of course). btw, little confused by this: > [RC languages?] have to run expensive compaction and locking steps Was this meant to refer to GC languages? I've never seen a non-GC language that does compaction (we're talking about the same thing? moving all the heap allocations around to reduce fragmentation?) That's more of a GC language thing that e.g. Java does; it would require the language/runtime to be able to precisely spot pointers and be able to throw in a trap when one it's changing is dereferenced or to entirely stop the world first, and I don't know why you'd take on that complexity and expense if you're not doing GC...
    - manwe150 3y ago
      Yes, rust is a great language. I love what it does. It also has memory leaks, because it is missing the cleanup part of the GC so it can’t deal with reference counter cycles. Well, there is boehm if you do want to take on that complexity. But no, I am referring to the book keeping operations that are done internal to the malloc and free calls. They are largely hidden from you as the user of the API, but you still have to pay for them in runtime. I thought rust didn’t have any safe arena allocators, but that some people make them by tricking the system with indexes instead and such, though word was they might come soon https://news.ycombinator.com/item?id=33403324 https://news.ycombinator.com/item?id=33403324
scottlamb 3y ago
Are you interpreting "GC every heap allocation" as "each heap allocation causes an entire cycle of GC"? That's...not what that phrase means.
- charcircuit 3y ago
  Yes I am. Java's GC can stop the world at any safepoint. It isn't limited to when objects are allocated and it doesn't do it every time.
  - scottlamb 3y ago
    I'm aware. A cycle per allocation would be absurdly expensive. "GC every heap allocation" actually means that every heap allocation is subject to garbage collection and has to be traversed on every cycle in which it's reachable. If the GC algorithm uses compaction and/or generations, it gets moved around too. That all has a cost.

yencabulator 3y ago

The reason Arc hurts is that it's global and atomic. Multiple cores competing to inc/dec a shared reference count is pretty much a worst case scenario for modern CPUs. Under modern garbage collection, there's very little need to coordinate across cores, just some barriers every now and then to mark things safe.

scottlamb 3y ago
True, but impact depends on both total core count you're using in the process and what you're doing with Arc. I'm imagining Arc<RequestState> with low concurrency on each RequestState so impact of cache line bouncing should be negligible. If you're talking about Arc<GlobalStuff>, I'd probably use Box::leak instead. There's also a few crates that do epoch gc. I haven't really written super high request rate multicore stuff in Rust, but I have in C++, and there for global-ish occasionally updated config stuff, I used a epoch gc implementation on top of Linux rseq to totally eliminate the effect you're describing.
- yencabulator 3y ago
  The classic case where Arc hurts and GC really shines is Arc<Config>. Arc makes otherwise read-only activity write.
  - pkolaczk 3y ago
    How many independent things do you need to access the config? A dozen? Then there is a dozen calls to inc the RC at startup, but there is no later cost to read from object behind Arc. So you may read your config a million times and Arc changes nothing. While GC will have to scan the Config structure every time a full GC is needed, so there is an added cost.
    - yencabulator 3y ago
      In the trouble scenario, config may change at any time. A getter function has to return a freshly-incremented Arc<Config>, which lives for roughly one request.