3 ms·

In Rust code, if you're not able to locally reason that an array index is valid, it should be written with .get() and the None case handled appropriately. It's

In Rust code, if you're not able to locally reason that an array index is valid, it should be written with .get() and the None case handled appropriately. It's impossible to claim there's "no actual memory safety issue" when a program's invariants have been broken: all bets are off at that point.

megous 4y ago
Why allow indexed access at all if the compiler is emitting a conditional check anyway?
- vgel 4y ago
  Some people have argued that indexed access is a wart, but it would be quite heavyweight to always have to unwrap an option when accessing a known-good index: let foo = [0_u8, 1, 2]; foo[0].unwrap(); // really? Instead, indexing on arrays / vecs is (essentially) sugar for .get(index).unwrap(), if you don't want the unwrap behavior use get. This is very similar to Python, though Python throws an exception which obviously isn't available to Rust.
  - couchand 4y ago
    But in real code you never want unwrap, so why provide sugar for it?
- connicpu 4y ago
  Because in the common case you assume that, if your code is correct, all of your indexing will be in bounds, but for memory safety reasons we need a bug to be reported if memory safety would have been violated. So we allow direct indexing with a panic on out of bounds because it's the most ergonomic for that common case
  - megous 4y ago
    The bug is not just reported here, the whole computer shuts down and all your unsaved work gets lost. That's not very ergonomic either.
  - couchand 4y ago
    I've come to believe ergonomics is a siren song here, mostly because recently I've been considering panics as forbidden as memory unsafety is... it's never okay for your embedded system or web server to panic, so don't act like that style is somehow preferable. If you "know" the index is in bounds, you can get_unchecked. Otherwise you should get. Either would be a sane choice for the index operator.
- nextaccountic 4y ago
  Because it's convenient and familiar to most programmers. Not providing bounds-checked indexing makes some kinds of code very hard to write. But note his problem also happens with integer division. In Rust, a[x] on an array or vec is really a roughly a shortand for a.get(x).unwrap() (with a different error message) Likewise, a / b on integers is a kind of a shortand for a.checked_div(b).unwrap() The thing is, if the index ever is out of bounds, or if the denominator is zero, the program has a bug, 100% of time. And if you catch a bug using an assertion there is seldom anything better than interrupting the execution (the only thing I can think of is restarting the program or the subsystem). If you continue execution past a programming error, you may sometimes corrupt data structures or introduce bizarre, hard to debug situations. Doing a pattern match on a.get(x) doesn't help because if it's ever None (and your program logic expects that x is in bounds) then you are kind of forced to bail. The downside here is that we aren't catching this bug at compile time. And it's true that sometimes we can rewrite the program to not have an indexing operation, usually using iterators (eliding the bounds check will make the program run faster, too). But in general this is not possible, at least not without bringing formal methods. But that's what tests are for, to ensure the correctness of stuff type errors can't catch. Now, there are some crates like https://github.com/dtolnay/no-panic https://github.com/dtolnay/no-panic or https://github.com/facebookexperimental/MIRAI https://github.com/facebookexperimental/MIRAI that will check that your code is panic free. The first one is based on the fact that llvm optimizations can often remove dead code and thus remove the panic from a[x] or a / b - if it doesn't, then compilation fails. The second one employs formal methods to mathematically prove that there is no panic. I guess those techniques will eventually be ported to the kernel even if panics happen differently there (by hooking on the BUG mechanism or whatever)
AshamedCaptain 4y ago
> It's impossible to claim there's "no actual memory safety issue" when a program's invariants have been broken: all bets are off at that point. When the underlying _runtime's_ invariants have been broken, not when the program invariants have been broken. i.e. you can recover from almost everything save for a VM error in a VM language like Java, since there's no way for the program to mess up the VM's data structures in a way that they cannot be brought back to a defined state.
- couchand 4y ago
  let innocent_var = operation_that_panics_but_returns_something_random_instead(); unsafe { do_something_assuming_validity(innocent_var); }

megous 4y ago

Why allow indexed access at all if the compiler is emitting a conditional check anyway?

vgel 4y ago
Some people have argued that indexed access is a wart, but it would be quite heavyweight to always have to unwrap an option when accessing a known-good index: let foo = [0_u8, 1, 2]; foo[0].unwrap(); // really? Instead, indexing on arrays / vecs is (essentially) sugar for .get(index).unwrap(), if you don't want the unwrap behavior use get. This is very similar to Python, though Python throws an exception which obviously isn't available to Rust.
- couchand 4y ago
  But in real code you never want unwrap, so why provide sugar for it?
connicpu 4y ago
Because in the common case you assume that, if your code is correct, all of your indexing will be in bounds, but for memory safety reasons we need a bug to be reported if memory safety would have been violated. So we allow direct indexing with a panic on out of bounds because it's the most ergonomic for that common case
- megous 4y ago
  The bug is not just reported here, the whole computer shuts down and all your unsaved work gets lost. That's not very ergonomic either.
- couchand 4y ago
  I've come to believe ergonomics is a siren song here, mostly because recently I've been considering panics as forbidden as memory unsafety is... it's never okay for your embedded system or web server to panic, so don't act like that style is somehow preferable. If you "know" the index is in bounds, you can get_unchecked. Otherwise you should get. Either would be a sane choice for the index operator.
nextaccountic 4y ago
Because it's convenient and familiar to most programmers. Not providing bounds-checked indexing makes some kinds of code very hard to write. But note his problem also happens with integer division. In Rust, a[x] on an array or vec is really a roughly a shortand for a.get(x).unwrap() (with a different error message) Likewise, a / b on integers is a kind of a shortand for a.checked_div(b).unwrap() The thing is, if the index ever is out of bounds, or if the denominator is zero, the program has a bug, 100% of time. And if you catch a bug using an assertion there is seldom anything better than interrupting the execution (the only thing I can think of is restarting the program or the subsystem). If you continue execution past a programming error, you may sometimes corrupt data structures or introduce bizarre, hard to debug situations. Doing a pattern match on a.get(x) doesn't help because if it's ever None (and your program logic expects that x is in bounds) then you are kind of forced to bail. The downside here is that we aren't catching this bug at compile time. And it's true that sometimes we can rewrite the program to not have an indexing operation, usually using iterators (eliding the bounds check will make the program run faster, too). But in general this is not possible, at least not without bringing formal methods. But that's what tests are for, to ensure the correctness of stuff type errors can't catch. Now, there are some crates like https://github.com/dtolnay/no-panic https://github.com/dtolnay/no-panic or https://github.com/facebookexperimental/MIRAI https://github.com/facebookexperimental/MIRAI that will check that your code is panic free. The first one is based on the fact that llvm optimizations can often remove dead code and thus remove the panic from a[x] or a / b - if it doesn't, then compilation fails. The second one employs formal methods to mathematically prove that there is no panic. I guess those techniques will eventually be ported to the kernel even if panics happen differently there (by hooking on the BUG mechanism or whatever)

AshamedCaptain 4y ago

> It's impossible to claim there's "no actual memory safety issue" when a program's invariants have been broken: all bets are off at that point. When the underlying _runtime's_ invariants have been broken, not when the program invariants have been broken. i.e. you can recover from almost everything save for a VM error in a VM language like Java, since there's no way for the program to mess up the VM's data structures in a way that they cannot be brought back to a defined state.

couchand 4y ago
let innocent_var = operation_that_panics_but_returns_something_random_instead(); unsafe { do_something_assuming_validity(innocent_var); }