3 ms·

Usually you'd want to write almost all your slice or other container iterations with iterators, in a functional style. For the 5% of cases that are too complex

by tux3 11mo ago

Usually you'd want to write almost all your slice or other container iterations with iterators, in a functional style. For the 5% of cases that are too complex for standard iterators? I never bother justifying why my indexes are correct, but I don't see why not. You very rarely need SAFETY comments in Rust because almost all the code you write is safe in the first place. The language also gives you the tool to avoid manual iteration (not just for safety, but because it lets the compiler eliminate bounds checks), so it would actually be quite viable to write these comments, since you only need them when you're doing something unusual.

dist1ll 11mo ago
For iteration, yes. But there's other cases, like any time you have to deal with lots of linked data structures. If you need high performance, chances are that you'll have to use an index+arena strategy. They're also common in mathematical codebases.
wrs 11mo ago
I didn't restate the context from the code we're discussing: it must not panic. If you don't care if the code panics, then go ahead and unwrap/expect/index, because that conforms to your chosen error handling scheme. This is fine for lots of things like CLI tools or isolated subprocesses, and makes review a lot easier. So: first, identify code that cannot be allowed to panic. Within that code, yes, in the rare case that you use [i], you need to at least try to justify why you think it'll be in bounds. But it would be better not to. There are a couple of attempts at getting the compiler to prove that code can't panic (e.g., the no-panic crate).
- kibwen 11mo ago
  Indexing is comparatively rare given the existence of iterators, IMO. If your goal is to avoid any potential for panicking, I think you'd have a harder time with arithmetic overflow.
- echelon 11mo ago
  Cargo needs to grow a label for crates that provably do not panic. (Neverminding allocations and things outside our control flow.) I want to ban crates that panic from my dependency chain. The language could really use an extra set of static guarantees around this. I would opt in.
  - phire 11mo ago
    I think I'd prefer a compile-time guarantee. Something that allows me to tag annotate a function (or my whole crate) as "no panic", and get a compile error if the function or anything it calls has a reachable panic. This will allow it to work with many unmodified crates, as long as constant propagation can prove that any panics are unreachable. This approach will also allow crates to provide panicking and non panicking versions of their API (which many already do).
    - aw1621107 11mo ago
      I think the most common solution at the moment is dtolnay's no_panic [0]. That has a bunch of caveats, though, and the ergonomics leave something to be desired, so a first-party solution would probably be preferable. [0]: https://github.com/dtolnay/no-panic https://github.com/dtolnay/no-panic
    - echelon 11mo ago
      Yes, I want that. I also want to be able to (1) statically apply a badge on every crate that makes and meets these guarantees (including transitively with that crate's own dependencies) so I can search crates.io for stronger guarantees and (2) annotate my Cargo.toml to not import crates that violate this, so time isn't wasted compiling - we know it'll fail in advance. On the subject of this, I want more ability to filter out crates in our Cargo.toml. Such as a max dependency depth. Or a frozen set of dependencies that is guaranteed not to change so audits are easier. (Obviously we could vendor the code in and be in charge of our own destiny, but this feels like something we can let crate authors police.)
  - lenkite 11mo ago
    > I want to ban crates that panic from my dependency chain. Which means banning anything that allocates memory and thousands of stdlib functions/methods.
- lenkite 11mo ago
  What about memory allocation - how will you stop that from panicking ? `Vec::resize` will always panic in Rust. And this is just one example out of thousands in the Rust stdlib. Unless the language addresses no-panic in its governing design or allows try-catch, not sure how you go about this.
  - wrs 11mo ago
    That is slowly being addressed, but meanwhile it’s likely you have a reliable upper bound on how much heap your service needs, so it’s a much smaller worry. There are also techniques like up-front or static allocation if you want to make more certain.
    - NewJazz 11mo ago
      Yep and this postmortem details how their proxy modules use static allocation.
    - echelon 11mo ago
      I'm far more worried about some dependency calling unwrap() or expect() now. https://github.com/search?q=unwrap%28%29+language%3ARust&type=code https://github.com/search?q=unwrap%28%29+language%3ARust&typ... This is ridiculous. We're probably going to start seeing more of these. This was just the first, big highly visible instance. We should have a name for this similar to "my code just NPE'd". I suggest "unwrapped", as in, "My Rust app just unwrapped a present." I think we should start advocating for the deprecation and eventual removal of the unwrap/expect family of methods. There's no reason engineers shouldn't be handling Options and Results gracefully, either passing the state to the caller or turning to a success or fail path. Not doing this is just laziness.
  - assbuttbuttass 11mo ago
    In TFA they mentioned they preallocate all the memory up front

by tux3 11mo ago

dist1ll 11mo ago

For iteration, yes. But there's other cases, like any time you have to deal with lots of linked data structures. If you need high performance, chances are that you'll have to use an index+arena strategy. They're also common in mathematical codebases.

wrs 11mo ago

I didn't restate the context from the code we're discussing: it must not panic. If you don't care if the code panics, then go ahead and unwrap/expect/index, because that conforms to your chosen error handling scheme. This is fine for lots of things like CLI tools or isolated subprocesses, and makes review a lot easier. So: first, identify code that cannot be allowed to panic. Within that code, yes, in the rare case that you use [i], you need to at least try to justify why you think it'll be in bounds. But it would be better not to. There are a couple of attempts at getting the compiler to prove that code can't panic (e.g., the no-panic crate).

kibwen 11mo ago
Indexing is comparatively rare given the existence of iterators, IMO. If your goal is to avoid any potential for panicking, I think you'd have a harder time with arithmetic overflow.
echelon 11mo ago
Cargo needs to grow a label for crates that provably do not panic. (Neverminding allocations and things outside our control flow.) I want to ban crates that panic from my dependency chain. The language could really use an extra set of static guarantees around this. I would opt in.
- phire 11mo ago
  I think I'd prefer a compile-time guarantee. Something that allows me to tag annotate a function (or my whole crate) as "no panic", and get a compile error if the function or anything it calls has a reachable panic. This will allow it to work with many unmodified crates, as long as constant propagation can prove that any panics are unreachable. This approach will also allow crates to provide panicking and non panicking versions of their API (which many already do).
  - aw1621107 11mo ago
    I think the most common solution at the moment is dtolnay's no_panic [0]. That has a bunch of caveats, though, and the ergonomics leave something to be desired, so a first-party solution would probably be preferable. [0]: https://github.com/dtolnay/no-panic https://github.com/dtolnay/no-panic
  - echelon 11mo ago
    Yes, I want that. I also want to be able to (1) statically apply a badge on every crate that makes and meets these guarantees (including transitively with that crate's own dependencies) so I can search crates.io for stronger guarantees and (2) annotate my Cargo.toml to not import crates that violate this, so time isn't wasted compiling - we know it'll fail in advance. On the subject of this, I want more ability to filter out crates in our Cargo.toml. Such as a max dependency depth. Or a frozen set of dependencies that is guaranteed not to change so audits are easier. (Obviously we could vendor the code in and be in charge of our own destiny, but this feels like something we can let crate authors police.)
- lenkite 11mo ago
  > I want to ban crates that panic from my dependency chain. Which means banning anything that allocates memory and thousands of stdlib functions/methods.
lenkite 11mo ago
What about memory allocation - how will you stop that from panicking ? `Vec::resize` will always panic in Rust. And this is just one example out of thousands in the Rust stdlib. Unless the language addresses no-panic in its governing design or allows try-catch, not sure how you go about this.
- wrs 11mo ago
  That is slowly being addressed, but meanwhile it’s likely you have a reliable upper bound on how much heap your service needs, so it’s a much smaller worry. There are also techniques like up-front or static allocation if you want to make more certain.
  - NewJazz 11mo ago
    Yep and this postmortem details how their proxy modules use static allocation.
  - echelon 11mo ago
    I'm far more worried about some dependency calling unwrap() or expect() now. https://github.com/search?q=unwrap%28%29+language%3ARust&type=code https://github.com/search?q=unwrap%28%29+language%3ARust&typ... This is ridiculous. We're probably going to start seeing more of these. This was just the first, big highly visible instance. We should have a name for this similar to "my code just NPE'd". I suggest "unwrapped", as in, "My Rust app just unwrapped a present." I think we should start advocating for the deprecation and eventual removal of the unwrap/expect family of methods. There's no reason engineers shouldn't be handling Options and Results gracefully, either passing the state to the caller or turning to a success or fail path. Not doing this is just laziness.
- assbuttbuttass 11mo ago
  In TFA they mentioned they preallocate all the memory up front