4 ms·

Oh, that's interesting. I follow rust development with interest but have only built a few toy projects in myself and was definitely under the impression that th

Oh, that's interesting. I follow rust development with interest but have only built a few toy projects in myself and was definitely under the impression that these sorts of issues resided solely in `unsafe { ... }` blocks. But, yeah, with this example it makes sense. I'm relieved to hear that at least it doesn't pollute beyond module boundaries. But now I'm questioning what the difference is between `unsafe {...}` code and within-module code that accesses it, if the latter also has to deal with memory safety issues? Should `unsafe` only be declared at the module level then?

pcwalton 11y ago
No, I don't think that it should. Note that the only reason why the semantic invariant that len/cap aren't mutated is relevant to type safety is because the unsafe blocks depend on that to do the right thing. The reason for the phenomenon described in the article is not that it's possible to do unsafe things outside unsafe blocks. It's that, in practice, unsafe blocks often depend on invariants that are only enforced by module-level privacy.
derefr 11y ago
The idea with unsafe{} is that, while you might temporarily break an invariant (which Rust assumes for its memory-safety) while inside the unsafe{} block, you will restore that invariant before the end of the block. In other words, when you use unsafe{} code idiomatically, there shouldn't be anything like uninitialized-yet-accessible memory buffers "leaked out of" the unsafe{} code into safe code, even within the same module. The safe code should only see things it's possible to get from talking to safe code. The problem, of course, is that there's no way to enforce only this idiomatic (and rather strict) usage of unsafe{}. Well, that, and there may be some data structures that are fully impossible to implement with their promised complexity guarantees if you have to restore memory-safety invariants after every atomic step. This is justification enough for leaving unsafe{} the way it is, from my perspective; this is the same "leave it to the professionals" kind of code you tend to see in every standard library's data structure implementations, and not at all similar to the kind of code you'll ever need to write yourself (in 99% of cases.) I guess there's a lesson there, though: no, Rust will not magically protect you any more than C will when you wade into the "thorns" of domains like scientific computing or GIS. If you're implementing data structures that "must" reveal dirty or indeterminate state to the world to achieve full efficiency, there's not much you can do to be "fully" memory-safe. You can be safe at the higher level of your module boundary, but there's really no way to achieve the same elegant lexical bound on your invariants that you can with idiomatic unsafe{} code. When you leak things out of an unsafe{} block, you're taking the type system into your own hands, and it's up to you to re-establish the invariant at all dynamic module entry- and exit-points.
fmstephe 11y ago
"I'm relieved to hear that at least it doesn't pollute beyond module boundaries." This isn't true, it is easy to come up with an example where lack of safety spreads past module boundaries. impl Vec<T> { pub fn evil(&mut self, len: i32, cap: i32) { self.len = len; self.cap = cap; } } If we want to know our program is broken now, we need to look across all of the source code to see if evil is called in breaking ways. The point was that Rust provides abstraction facilities which allow us to contain these invariants. If we are careful.
- ralfj 11y ago
  This is not a counter-example to the claim you are citing. If you added this form of `evil` to `Vec`, then since `evil` is within the module, the proof of correctness has to cover `evil`. Doing that proof will fail. So, it is not necessary to look outside of the module to notice that something is wrong. Of course, if `len` were public, that would be a different story. Unsafety doesn't pollute beyond module boundaries if all the extra invariants are about private fields.
- Manishearth 11y ago
  `len` and `cap` are private fields and can't be accessed outside the module boundary, even by an impl.
  - phaylon 11y ago
    I've been wondering if it would make sense to encapsulate internal fields that are part of the unsafe machinery in a type forcing you to use unsafe blocks, even inside the type impl. [edit: Removed note about UnsafeCell since it's not really like the thing described above] I haven't really played with it though, most of my unsafe use is FFI.
    - Manishearth 11y ago
      You can, but IMO it's not so necessary. The module boundary thing doesn't cause many issues in practice is straightforward to reason about if you're aware of it. (And if you weren't aware of it you would probably not use `UnsafeField` either, since the compiler can't force you to do that)
kazagistar 11y ago
It could easily go past module boundaries. The point of the article is that the "unsafety" extends however far the invariants the unsafe code depend on extend. I could have an unsafe function that depends on a set being non-empty. And it could share that set with, say, a factory in a different module. So someone could mess with that code and cause the invariants to be violated, and suddenly the code is broken. Unless you have a language that is capable of expressing proofs (ie, Idris) you have to depend on good documentation and careful/tasteful design to prevent this kind of problem.

pcwalton 11y ago

No, I don't think that it should. Note that the only reason why the semantic invariant that len/cap aren't mutated is relevant to type safety is because the unsafe blocks depend on that to do the right thing. The reason for the phenomenon described in the article is not that it's possible to do unsafe things outside unsafe blocks. It's that, in practice, unsafe blocks often depend on invariants that are only enforced by module-level privacy.

derefr 11y ago

The idea with unsafe{} is that, while you might temporarily break an invariant (which Rust assumes for its memory-safety) while inside the unsafe{} block, you will restore that invariant before the end of the block. In other words, when you use unsafe{} code idiomatically, there shouldn't be anything like uninitialized-yet-accessible memory buffers "leaked out of" the unsafe{} code into safe code, even within the same module. The safe code should only see things it's possible to get from talking to safe code. The problem, of course, is that there's no way to enforce only this idiomatic (and rather strict) usage of unsafe{}. Well, that, and there may be some data structures that are fully impossible to implement with their promised complexity guarantees if you have to restore memory-safety invariants after every atomic step. This is justification enough for leaving unsafe{} the way it is, from my perspective; this is the same "leave it to the professionals" kind of code you tend to see in every standard library's data structure implementations, and not at all similar to the kind of code you'll ever need to write yourself (in 99% of cases.) I guess there's a lesson there, though: no, Rust will not magically protect you any more than C will when you wade into the "thorns" of domains like scientific computing or GIS. If you're implementing data structures that "must" reveal dirty or indeterminate state to the world to achieve full efficiency, there's not much you can do to be "fully" memory-safe. You can be safe at the higher level of your module boundary, but there's really no way to achieve the same elegant lexical bound on your invariants that you can with idiomatic unsafe{} code. When you leak things out of an unsafe{} block, you're taking the type system into your own hands, and it's up to you to re-establish the invariant at all dynamic module entry- and exit-points.

fmstephe 11y ago

"I'm relieved to hear that at least it doesn't pollute beyond module boundaries." This isn't true, it is easy to come up with an example where lack of safety spreads past module boundaries. impl Vec<T> { pub fn evil(&mut self, len: i32, cap: i32) { self.len = len; self.cap = cap; } } If we want to know our program is broken now, we need to look across all of the source code to see if evil is called in breaking ways. The point was that Rust provides abstraction facilities which allow us to contain these invariants. If we are careful.

ralfj 11y ago
This is not a counter-example to the claim you are citing. If you added this form of `evil` to `Vec`, then since `evil` is within the module, the proof of correctness has to cover `evil`. Doing that proof will fail. So, it is not necessary to look outside of the module to notice that something is wrong. Of course, if `len` were public, that would be a different story. Unsafety doesn't pollute beyond module boundaries if all the extra invariants are about private fields.
Manishearth 11y ago
`len` and `cap` are private fields and can't be accessed outside the module boundary, even by an impl.
- phaylon 11y ago
  I've been wondering if it would make sense to encapsulate internal fields that are part of the unsafe machinery in a type forcing you to use unsafe blocks, even inside the type impl. [edit: Removed note about UnsafeCell since it's not really like the thing described above] I haven't really played with it though, most of my unsafe use is FFI.
  - Manishearth 11y ago
    You can, but IMO it's not so necessary. The module boundary thing doesn't cause many issues in practice is straightforward to reason about if you're aware of it. (And if you weren't aware of it you would probably not use `UnsafeField` either, since the compiler can't force you to do that)

kazagistar 11y ago

It could easily go past module boundaries. The point of the article is that the "unsafety" extends however far the invariants the unsafe code depend on extend. I could have an unsafe function that depends on a set being non-empty. And it could share that set with, say, a factory in a different module. So someone could mess with that code and cause the invariants to be violated, and suddenly the code is broken. Unless you have a language that is capable of expressing proofs (ie, Idris) you have to depend on good documentation and careful/tasteful design to prevent this kind of problem.