5 ms·

> Seasoned Rust coders don’t spend time fighting the borrow checker My experience is that what makes your statement true, is that _seasoned_ Rust developers ju

> Seasoned Rust coders don’t spend time fighting the borrow checker My experience is that what makes your statement true, is that _seasoned_ Rust developers just sprinkle `Arc` all over the place, thus effectively switching to automatic garbage collection. Because 1) statically checked memory management is too restrictive for most kinds of non trivial data structures, and 2) the hoops of lifetimes you have to go to to please the static checker whenever you start doing anything non trivial are just above human comprehension level.

steveklabnik 1y ago
The only time I use Arc is wrapping contexts for web handlers. That doesn’t mean there aren’t other legitimate use cases, but “all the time” is not representative of the code I read or write, personally.
amw-zero 1y ago
How often are you writing non-trivial data structures?
swiftcoder 1y ago
I don't think there are any Arcs in my codebase (apart from a couple of regrettable ones needed to interface with Javascript callbacks in WASM - this is more a WASM problem than a rust problem).
- ChadNauseam 1y ago
  haha, I was about to leave the exact same comment. how are you finding wasm? I’ve been feeling like rust+react is my new favorite tech stack
  - swiftcoder 1y ago
    I love it, but I'm mainly using it for webgl/webgpu stuff, so relatively little interaction with the DOM - I feel like DOM interaction is still kind of painful through rust/wasm
dev_l1x_be 1y ago
I am not sure this is true. Maybe with shared async access it is. I rarely use Arc.
qw3rty01 1y ago
This is exactly the opposite of what he’s saying, using Arc everywhere is hacking around the borrow checker, a seasoned rust developer will structure their code in a way that works with the borrow checker; Arc has a very specific use case and a seasoned rust developer will rarely use it
- jasonjmcghee 1y ago
  This is awkward. I've written a fair amount of rust. I reach for Arc frequently. I see the memory layout implications now. Do you tend to use a lot of Arenas?
  - dminik 1y ago
    I've not explored every program domain, but in general I see two kinds of program memory access patterns. The first is a fairly generic input -> transform -> output. This is your generic request handler for instance. You receive a payload, run some transform on that (and maybe a DB request) and then produce a response. In this model, Arc is very fitting for some shared (im)mutable state. Like DB connections, configuration and so on. The second pattern is something like: state + input -> transform -> new state. Eg. you're mutating your app state based on some input. This fits stuff like games, but also retained UIs, programming language interpreters and so on on. Using ARCs here muddles the ownership. The gamedev ecosystem has found a way to manage this by employing ECS, and while it can be overkill, the base DOD principles can still be very helpful. Treat your data as what it is; data. Use indices/keys instead of pointers to represent relations. Keep it simple. Arenas can definitely be a part of that solution.
- packetlost 1y ago
  Arc<T> is all over the place if you're writing async code unfortunately. IMO Tokio using a work-stealing threaded scheduler by default and peppering literally everything with Send + Sync constraints was a huge misstep.
  - vlovich123 1y ago
    Arc + work stealing scheduler is common. But work stealing schedulers are common (eg libdispatch popularized it). I believe the only alternative is thread-per core but they’re not very common/popular. For what it’s worth zig would look very similar except their novel injectable I/O syntax isn’t compatible with work stealing. Even then, I’d agree that while Arc is used in lots of places in work stealing runtimes, I disagree that it’s used everywhere or that you can really do anything else if you want to leverage all your cores with minimum effort and not having to build your application specialized to deal with that.
kibwen 1y ago
> _seasoned_ Rust developers just sprinkle `Arc` all over the place No, this couldn't be further from the truth.
- 9rx 1y ago
  If they aren't sprinkling `Arc` all over, what are they seasoning with instead?
  - veber-alex 1y ago
    'a
    - 9rx 1y ago
      Can't be. Lifetime annotations are present in unseasoned Rust. The question was about what seasoning are being added, if not `Arc`?
  - oconnor663 1y ago
    Indexes: https://jacko.io/object_soup.html https://jacko.io/object_soup.html
andrewl-hn 1y ago
`Arc`s show up all over the place specifically in async code that targets Tokio runtime running in multithreaded mode. Mostly this is because `tokio::spawn` requires `Future`s to be `Send + 'static`, and this function is a building block of most libraries and frameworks built on top of Tokio. If you use Rust for web server backend code then yes, you see `Arc`s everywhere. Otherwise their use is pretty rare, even in large projects. Rust is somewhat unique in that regard, because most Rust code that is written is not really a web backend code.
- khuey 1y ago
  > `Arc`s show up all over the place specifically in async code that targets Tokio runtime running in multithreaded mode. Mostly this is because `tokio::spawn` requires `Future`s to be `Send + 'static`, and this function is a building block of most libraries and frameworks built on top of Tokio. To some extent this is unavoidable. Non-'static lifetimes correspond (roughly) to a location on the program stack. Since a Future that suspends can't reasonably stay on the stack it can't have a lifetime other than 'static. Once it has to be 'static, it can't borrow anything (that's not itself 'static), so you either have to Copy your data or Rc/Arc it. This, btw, is why even tokio's spawn_local has a 'static bound on the Future. It would be nice if it were ergonomic for library authors to push the decision about whether to use Rc<RefCell<T>> or Arc<Mutex<T>> (which are non-threadsafe and threadsafe variants of the same underlying concept) to the library consumer.
  - bryal 1y ago
    Non-tokio runtimes manage just fine without all of those bounds for local, single-threaded execution though. https://docs.rs/smol/latest/smol/fn.block_on.html https://docs.rs/smol/latest/smol/fn.block_on.html
    - khuey 1y ago
      Blocking on an asynchronous task and waiting for it to finish is fundamentally a different operation than spawning an asynchronous operation and returning to the caller to execute entirely different code. smol's spawn also requires the Future to be 'static (https://docs.rs/smol/latest/smol/fn.spawn.html https://docs.rs/smol/latest/smol/fn.spawn.html), while tokio's local block_on also does not require 'static or Send + Sync (https://docs.rs/tokio/latest/tokio/task/struct.LocalSet.html#method.block_on https://docs.rs/tokio/latest/tokio/task/struct.LocalSet.html...).
hu3 1y ago
I did some quick search, not sure if this supports or denies your point: - 151 instances of "Arc<" in Servo: https://github.com/search?q=repo%3Aservo%2Fservo+Arc%3C&type=code https://github.com/search?q=repo%3Aservo%2Fservo+Arc%3C&type... - 5 instances of "Arc<" in AWS SDK for Rust https://github.com/search?q=repo%3Arusoto%2Frusoto%20Arc%3C&type=code https://github.com/search?q=repo%3Arusoto%2Frusoto%20Arc%3C&... - 0 instances for "Arc<" in LOC https://github.com/search?q=repo%3Acgag%2Floc%20Arc%3C&type=code https://github.com/search?q=repo%3Acgag%2Floc%20Arc%3C&type=...
- qzw 1y ago
  Appreciate the real numbers. Would be interesting to see what percentage of data structures contain Arc, but that's a lot more work.
- tptacek 1y ago
  Why would you expect the AWS SDK to have complicated memory management?
  - hu3 1y ago
    I don't? Those were just from a quick search and I didn't want to cherrypick either way.
    - tptacek 1y ago
      OK. I was just caught out because you'd expect the AWS SDK to be distinctively simple (at least relative to its size). It's an HTTP API wrapper.
- liuliu 1y ago
  - 454 instances of "Rc<" in Servo: https://github.com/search?q=repo%3Aservo%2Fservo+Rc%3C&type=code https://github.com/search?q=repo%3Aservo%2Fservo+Rc%3C&type=... - 6 instances of "Rc<" in AWS SDK for Rust: https://github.com/search?q=repo%3Arusoto%2Frusoto+Rc%3C&type=code https://github.com/search?q=repo%3Arusoto%2Frusoto+Rc%3C&typ... - 0 instance for "Rc<" in LOC: https://github.com/search?q=repo%3Acgag%2Floc+Rc%3C&type=code https://github.com/search?q=repo%3Acgag%2Floc+Rc%3C&type=cod... (Disclaimer: I don't know what these repos are except Servo).
levkk 1y ago
Definitely not. Arc is for immutable (or sync, e.g. atomics, mutexes) data, while borrow checker protects against concurrent mutations. I think you meant Arc<Mutex<T>> everywhere, but that code smells immediately and seasoned Rust devs don't do that.
Aurornis 1y ago
> thus effectively switching to automatic garbage collection Arc isn't really garbage collection. It's like a reference counted smart pointer like C++ has shared_ptr. If you drop an Arc and it's the last reference to the underlying object, it gets dropped deterministically. Garbage collection generally refers to more complex systems that periodically identify and free unused objects in a less deterministic manner.
- jcelerier 1y ago
  > Arc isn't really garbage collection. It's like a reference counted smart pointer like C++ has shared_ptr. In c++ land this is very often called garbage collection too
- jandrewrogers 1y ago
  This still raises the question of why Arc is purportedly used so heavily. I've written 100s of kLoC of modern systems C++ and never needed std::shared_ptr.
  - pjmlp 1y ago
    For the same reason Unreal uses one. Large scale teams always get pointer ownership wrong. Project Zero has enough examples.
- Arnavion 1y ago
  >Garbage collection generally refers to more complex systems that periodically identify and free unused objects in a less deterministic manner. No, this is a subset of garbage collection called tracing garbage collection. "Garbage collection" absolutely includes refcounting.
  - simonask 1y ago
    There’s just no good reason to conflate the two. Rust’s Arc and C++’s std::shared_ptr do not reclaim reference cycles, so you can call it “garbage collection” if you want, but the colloquial understanding is way more useful.
kannanvijayan 1y ago
Not sure how seasoned I am, but I reject any comparison to a cooking utensil! I do find myself running into lifetime and borrow-checker issues much less these days when writing larger programs in rust. And while your comment is a bit cheeky, I think it gets at something real. One of the implicit design mentalities that develops once you write rust for a while is a good understanding of where to apply the `UnsafeCell`-related types, which includes `Arc` but also `Rc` and `RefCell` and `Cell`. These all relate to inner mutability, and there are many situations where plopping in the right one of these effectively resolves some design requirement. The other idiomatic thing that happens is that you implicitly begin structuring your abstract data layouts in terms of thunks of raw structured data and connections between them. This usually involves an indirection - i.e. you index into an array of things instead of holding a pointer to the thing. Lastly, where lifetimes do get involved, you tend to have a prior idea of what thing they annotate. The example in the article is a good case study of that. The author is parsing a `.notes` file and building some index of it. The text of the `.notes` file is the obvious lifetime anchor here. You would write your indexing logic with one lifetime 'src: `fn build_index<'src>(src: &'src str)` Internally to the indexing code, references to 'src-annotated things can generally pass around freely as their lifetime converges after it. Externally to the indexing code you'd build a string of the notes text, and passing a reference to that to the `build_index` function. For simple CLI programs, you tend not to really need anything more than this. It gets more hairy if you're looking at constructing complex object graphs with complex intermediate state, partial construction of sub-states, etc. Keeping track of state that's valid at some level, while temporarily broken at another level, is where it gets really annoying with multiple nested lifetimes and careful annotation required. But it was definitely a bit of a hair-pulling journey to get to my state of quasi-peace with Rust's borrow checker.
bryanlarsen 1y ago
Or more likely, sprinkle .clone() liberally and Arc or an Arc wrapper (ArcSwap, tokio's watch channels, etc) strategically.
ViewTrick1002 1y ago
> My experience is that what makes your statement true, is that _seasoned_ Rust developers just sprinkle `Arc` all over the place, thus effectively switching to automatic garbage collection. How else would you safely share data in multi-threaded code? Which is the only reason to use Atomic reference counts.

steveklabnik 1y ago

The only time I use Arc is wrapping contexts for web handlers. That doesn’t mean there aren’t other legitimate use cases, but “all the time” is not representative of the code I read or write, personally.

amw-zero 1y ago

How often are you writing non-trivial data structures?

swiftcoder 1y ago

I don't think there are any Arcs in my codebase (apart from a couple of regrettable ones needed to interface with Javascript callbacks in WASM - this is more a WASM problem than a rust problem).

ChadNauseam 1y ago
haha, I was about to leave the exact same comment. how are you finding wasm? I’ve been feeling like rust+react is my new favorite tech stack
- swiftcoder 1y ago
  I love it, but I'm mainly using it for webgl/webgpu stuff, so relatively little interaction with the DOM - I feel like DOM interaction is still kind of painful through rust/wasm

dev_l1x_be 1y ago

I am not sure this is true. Maybe with shared async access it is. I rarely use Arc.

qw3rty01 1y ago

This is exactly the opposite of what he’s saying, using Arc everywhere is hacking around the borrow checker, a seasoned rust developer will structure their code in a way that works with the borrow checker; Arc has a very specific use case and a seasoned rust developer will rarely use it

jasonjmcghee 1y ago
This is awkward. I've written a fair amount of rust. I reach for Arc frequently. I see the memory layout implications now. Do you tend to use a lot of Arenas?
- dminik 1y ago
  I've not explored every program domain, but in general I see two kinds of program memory access patterns. The first is a fairly generic input -> transform -> output. This is your generic request handler for instance. You receive a payload, run some transform on that (and maybe a DB request) and then produce a response. In this model, Arc is very fitting for some shared (im)mutable state. Like DB connections, configuration and so on. The second pattern is something like: state + input -> transform -> new state. Eg. you're mutating your app state based on some input. This fits stuff like games, but also retained UIs, programming language interpreters and so on on. Using ARCs here muddles the ownership. The gamedev ecosystem has found a way to manage this by employing ECS, and while it can be overkill, the base DOD principles can still be very helpful. Treat your data as what it is; data. Use indices/keys instead of pointers to represent relations. Keep it simple. Arenas can definitely be a part of that solution.
packetlost 1y ago
Arc<T> is all over the place if you're writing async code unfortunately. IMO Tokio using a work-stealing threaded scheduler by default and peppering literally everything with Send + Sync constraints was a huge misstep.
- vlovich123 1y ago
  Arc + work stealing scheduler is common. But work stealing schedulers are common (eg libdispatch popularized it). I believe the only alternative is thread-per core but they’re not very common/popular. For what it’s worth zig would look very similar except their novel injectable I/O syntax isn’t compatible with work stealing. Even then, I’d agree that while Arc is used in lots of places in work stealing runtimes, I disagree that it’s used everywhere or that you can really do anything else if you want to leverage all your cores with minimum effort and not having to build your application specialized to deal with that.

kibwen 1y ago

> _seasoned_ Rust developers just sprinkle `Arc` all over the place No, this couldn't be further from the truth.

9rx 1y ago
If they aren't sprinkling `Arc` all over, what are they seasoning with instead?
- veber-alex 1y ago
  'a
  - 9rx 1y ago
    Can't be. Lifetime annotations are present in unseasoned Rust. The question was about what seasoning are being added, if not `Arc`?
- oconnor663 1y ago
  Indexes: https://jacko.io/object_soup.html https://jacko.io/object_soup.html

andrewl-hn 1y ago

`Arc`s show up all over the place specifically in async code that targets Tokio runtime running in multithreaded mode. Mostly this is because `tokio::spawn` requires `Future`s to be `Send + 'static`, and this function is a building block of most libraries and frameworks built on top of Tokio. If you use Rust for web server backend code then yes, you see `Arc`s everywhere. Otherwise their use is pretty rare, even in large projects. Rust is somewhat unique in that regard, because most Rust code that is written is not really a web backend code.

khuey 1y ago
> `Arc`s show up all over the place specifically in async code that targets Tokio runtime running in multithreaded mode. Mostly this is because `tokio::spawn` requires `Future`s to be `Send + 'static`, and this function is a building block of most libraries and frameworks built on top of Tokio. To some extent this is unavoidable. Non-'static lifetimes correspond (roughly) to a location on the program stack. Since a Future that suspends can't reasonably stay on the stack it can't have a lifetime other than 'static. Once it has to be 'static, it can't borrow anything (that's not itself 'static), so you either have to Copy your data or Rc/Arc it. This, btw, is why even tokio's spawn_local has a 'static bound on the Future. It would be nice if it were ergonomic for library authors to push the decision about whether to use Rc<RefCell<T>> or Arc<Mutex<T>> (which are non-threadsafe and threadsafe variants of the same underlying concept) to the library consumer.
- bryal 1y ago
  Non-tokio runtimes manage just fine without all of those bounds for local, single-threaded execution though. https://docs.rs/smol/latest/smol/fn.block_on.html https://docs.rs/smol/latest/smol/fn.block_on.html
  - khuey 1y ago
    Blocking on an asynchronous task and waiting for it to finish is fundamentally a different operation than spawning an asynchronous operation and returning to the caller to execute entirely different code. smol's spawn also requires the Future to be 'static (https://docs.rs/smol/latest/smol/fn.spawn.html https://docs.rs/smol/latest/smol/fn.spawn.html), while tokio's local block_on also does not require 'static or Send + Sync (https://docs.rs/tokio/latest/tokio/task/struct.LocalSet.html#method.block_on https://docs.rs/tokio/latest/tokio/task/struct.LocalSet.html...).

hu3 1y ago

I did some quick search, not sure if this supports or denies your point: - 151 instances of "Arc<" in Servo: https://github.com/search?q=repo%3Aservo%2Fservo+Arc%3C&type=code https://github.com/search?q=repo%3Aservo%2Fservo+Arc%3C&type... - 5 instances of "Arc<" in AWS SDK for Rust https://github.com/search?q=repo%3Arusoto%2Frusoto%20Arc%3C&type=code https://github.com/search?q=repo%3Arusoto%2Frusoto%20Arc%3C&... - 0 instances for "Arc<" in LOC https://github.com/search?q=repo%3Acgag%2Floc%20Arc%3C&type=code https://github.com/search?q=repo%3Acgag%2Floc%20Arc%3C&type=...

qzw 1y ago
Appreciate the real numbers. Would be interesting to see what percentage of data structures contain Arc, but that's a lot more work.
tptacek 1y ago
Why would you expect the AWS SDK to have complicated memory management?
- hu3 1y ago
  I don't? Those were just from a quick search and I didn't want to cherrypick either way.
  - tptacek 1y ago
    OK. I was just caught out because you'd expect the AWS SDK to be distinctively simple (at least relative to its size). It's an HTTP API wrapper.
liuliu 1y ago
- 454 instances of "Rc<" in Servo: https://github.com/search?q=repo%3Aservo%2Fservo+Rc%3C&type=code https://github.com/search?q=repo%3Aservo%2Fservo+Rc%3C&type=... - 6 instances of "Rc<" in AWS SDK for Rust: https://github.com/search?q=repo%3Arusoto%2Frusoto+Rc%3C&type=code https://github.com/search?q=repo%3Arusoto%2Frusoto+Rc%3C&typ... - 0 instance for "Rc<" in LOC: https://github.com/search?q=repo%3Acgag%2Floc+Rc%3C&type=code https://github.com/search?q=repo%3Acgag%2Floc+Rc%3C&type=cod... (Disclaimer: I don't know what these repos are except Servo).

levkk 1y ago

Definitely not. Arc is for immutable (or sync, e.g. atomics, mutexes) data, while borrow checker protects against concurrent mutations. I think you meant Arc<Mutex<T>> everywhere, but that code smells immediately and seasoned Rust devs don't do that.

Aurornis 1y ago

> thus effectively switching to automatic garbage collection Arc isn't really garbage collection. It's like a reference counted smart pointer like C++ has shared_ptr. If you drop an Arc and it's the last reference to the underlying object, it gets dropped deterministically. Garbage collection generally refers to more complex systems that periodically identify and free unused objects in a less deterministic manner.

jcelerier 1y ago
> Arc isn't really garbage collection. It's like a reference counted smart pointer like C++ has shared_ptr. In c++ land this is very often called garbage collection too
jandrewrogers 1y ago
This still raises the question of why Arc is purportedly used so heavily. I've written 100s of kLoC of modern systems C++ and never needed std::shared_ptr.
- pjmlp 1y ago
  For the same reason Unreal uses one. Large scale teams always get pointer ownership wrong. Project Zero has enough examples.
Arnavion 1y ago
>Garbage collection generally refers to more complex systems that periodically identify and free unused objects in a less deterministic manner. No, this is a subset of garbage collection called tracing garbage collection. "Garbage collection" absolutely includes refcounting.
- simonask 1y ago
  There’s just no good reason to conflate the two. Rust’s Arc and C++’s std::shared_ptr do not reclaim reference cycles, so you can call it “garbage collection” if you want, but the colloquial understanding is way more useful.

kannanvijayan 1y ago

Not sure how seasoned I am, but I reject any comparison to a cooking utensil! I do find myself running into lifetime and borrow-checker issues much less these days when writing larger programs in rust. And while your comment is a bit cheeky, I think it gets at something real. One of the implicit design mentalities that develops once you write rust for a while is a good understanding of where to apply the `UnsafeCell`-related types, which includes `Arc` but also `Rc` and `RefCell` and `Cell`. These all relate to inner mutability, and there are many situations where plopping in the right one of these effectively resolves some design requirement. The other idiomatic thing that happens is that you implicitly begin structuring your abstract data layouts in terms of thunks of raw structured data and connections between them. This usually involves an indirection - i.e. you index into an array of things instead of holding a pointer to the thing. Lastly, where lifetimes do get involved, you tend to have a prior idea of what thing they annotate. The example in the article is a good case study of that. The author is parsing a `.notes` file and building some index of it. The text of the `.notes` file is the obvious lifetime anchor here. You would write your indexing logic with one lifetime 'src: `fn build_index<'src>(src: &'src str)` Internally to the indexing code, references to 'src-annotated things can generally pass around freely as their lifetime converges after it. Externally to the indexing code you'd build a string of the notes text, and passing a reference to that to the `build_index` function. For simple CLI programs, you tend not to really need anything more than this. It gets more hairy if you're looking at constructing complex object graphs with complex intermediate state, partial construction of sub-states, etc. Keeping track of state that's valid at some level, while temporarily broken at another level, is where it gets really annoying with multiple nested lifetimes and careful annotation required. But it was definitely a bit of a hair-pulling journey to get to my state of quasi-peace with Rust's borrow checker.

bryanlarsen 1y ago

Or more likely, sprinkle .clone() liberally and Arc or an Arc wrapper (ArcSwap, tokio's watch channels, etc) strategically.

ViewTrick1002 1y ago

> My experience is that what makes your statement true, is that _seasoned_ Rust developers just sprinkle `Arc` all over the place, thus effectively switching to automatic garbage collection. How else would you safely share data in multi-threaded code? Which is the only reason to use Atomic reference counts.