3 ms·

https://internals.rust-lang.org/t/language-vision-regarding-safety-guarantees/24418 https://internals.rust-lang.org/t/language-vision-regarding-... You must re

https://internals.rust-lang.org/t/language-vision-regarding-safety-guarantees/24418 https://internals.rust-lang.org/t/language-vision-regarding-... You must reason about the invariants in unsafe code on a global level. In particular, you could have unsafe code in crate A, whose data are then used by crate B. It could be fine. But then crate B changes its implementation which now violates the invariant expectations of crate A.

pron 13d ago
This is true. In Java, we have a notion we call "integrity", which is a generalisation of memory safety and includes a host of properties guaranteed by the platform. It includes memory safety, but also things like "a non-public method cannot be called or a non-public field cannot be accessed (even reflectively) by code in another module". To address the problem that once integrity can be violated anywhere, only global analysis can prove that nothing bad happens, we've done two things: 1. We require the application to explicitly permit any integrity violation by a module; i.e. a library can't allow itself to violate integrity. This is a principle we call "Integrity by Default" (https://openjdk.org/jeps/8305968 https://openjdk.org/jeps/8305968). 2. We try to minimise the need for potential integrity violations (this is very different from Rust, which requires unsafe even for things like benign write/write races, which are fairly common, and various basic data structures). Over the years we've offered safe replacements for things that used to require Unsafe. In other words, clearly demarcating unsafe code isn't enough if it's needed at all in many situations. It isn't perfect, of course, as some libraries do require unsafe operations for direct interaction with native code or with memory, but their number has been greatly reduced, and they cannot do this without the application's explicit approval. Interestingly, this has annoyed library authors who want to do unsafe things but don't want to application authors to be alarmed because "we know what we're doing," and it's also annoyed some application authors who want to use such libraries and are forced to explicitly add permissions. But I think that the community, as a whole, has eventually accepted this because the harm done to those who don't care is small (they just need to add the permissions), to those who do care it helps a lot, and because fewer and fewer libraries require "integrity-busting" permissions, many applications need to do absolutely nothing and get important guarantees for free.
- aw1621107 13d ago
  > this is very different from Rust, which requires unsafe even for things like benign write/write races, which are fairly common, and various basic data structures I know this paper [0] is quite old at this point, but the mention of benign data races reminded me of it. Would you happen to know how applicable it is to modern memory models? [0]: https://www.usenix.org/legacy/event/hotpar11/tech/final_files/Boehm.pdf https://www.usenix.org/legacy/event/hotpar11/tech/final_file...
  - pron 12d ago
    Benign write/write races (when multiple threads do unordered writes of the same value to the same address) are quite common and useful, both in parallel algorithms and in lazy initialisation. Useful benign read/write races are far more rare to the point I'd say it's ok to assume they don't (or shouldn't) exist. However, in C and C++ (and Rust) benign non-atomic write/write races are UB (indeed, LLVM also treats them as potential causes of UB). In C# and in Java they are safe (although Java currently only has non-atomic writes on 32-bit machines, but soon they'll be more common when value types are enhanced). LLVM even has a specific construct to support the Java-style memory model (https://llvm.org/docs/Atomics.html#unordered https://llvm.org/docs/Atomics.html#unordered), and Zig lets you use it (https://ziglang.org/documentation/master/#atomicStore https://ziglang.org/documentation/master/#atomicStore).
aw1621107 13d ago
> In particular, you could have unsafe code in crate A, whose data are then used by crate B. Is this backwards? If B consumes data from A then to me that does not imply that A depends on anything from B; for a more concrete example that sentence reads to me like A is basically "throwing data over the wall" to B and whatever B does with said data is of no relevance to A. As a result, if B changes that shouldn't affect A. Also for what it's worth I get the impression you and treyd might be talking about slightly different things when talking about whether unsafe code composes. I believe treyd is referring to the RustBelt series of papers [0, 1], for which the statement "unsafe code composes" means (at a high level) that adding a module with a memory-safe API to a memory-safe system will result in a memory-safe system as long as the implementation upholds the safe semantics. Yes, the last bit can be a rather significant caveat, as you said. What you're talking about seems more along the lines of needing to look beyond the boundaries of unsafe blocks to prove that the unsafe block upholds its invariants, which is also true. I think you only need to check within whatever safe encapsulation boundary is relevant, though, rather than globally. [0]: https://people.mpi-sws.org/~dreyer/papers/rustbelt/paper.pdf https://people.mpi-sws.org/~dreyer/papers/rustbelt/paper.pdf [1]: https://plv.mpi-sws.org/rustbelt/rbrlx/paper.pdf https://plv.mpi-sws.org/rustbelt/rbrlx/paper.pdf
- toast0 12d ago
  > Is this backwards? If B consumes data from A then to me that does not imply that A depends on anything from B; for a more concrete example that sentence reads to me like A is basically "throwing data over the wall" to B and whatever B does with said data is of no relevance to A. As a result, if B changes that shouldn't affect A. This is a specifically crafted bad idea, but you could have module A use unsafe to craft a Vec<u8> that is safe to use to read or write, but not to grow or shrink. You declare an invariant that the receiver shalt not grow or shrink the Vec. If B only reads and write, you're good. But if a future B breaks the invariant, bad things happen. As I said, specifically a bad idea; there's a much better type to use if the thing can't grow or shrink... No real world example, because I don't think we've run into memory safety issues with unsafe in the Rust code base I work in... but we only use unsafe where it's required (syscalls and other FFI).

pron 13d ago

This is true. In Java, we have a notion we call "integrity", which is a generalisation of memory safety and includes a host of properties guaranteed by the platform. It includes memory safety, but also things like "a non-public method cannot be called or a non-public field cannot be accessed (even reflectively) by code in another module". To address the problem that once integrity can be violated anywhere, only global analysis can prove that nothing bad happens, we've done two things: 1. We require the application to explicitly permit any integrity violation by a module; i.e. a library can't allow itself to violate integrity. This is a principle we call "Integrity by Default" (https://openjdk.org/jeps/8305968 https://openjdk.org/jeps/8305968). 2. We try to minimise the need for potential integrity violations (this is very different from Rust, which requires unsafe even for things like benign write/write races, which are fairly common, and various basic data structures). Over the years we've offered safe replacements for things that used to require Unsafe. In other words, clearly demarcating unsafe code isn't enough if it's needed at all in many situations. It isn't perfect, of course, as some libraries do require unsafe operations for direct interaction with native code or with memory, but their number has been greatly reduced, and they cannot do this without the application's explicit approval. Interestingly, this has annoyed library authors who want to do unsafe things but don't want to application authors to be alarmed because "we know what we're doing," and it's also annoyed some application authors who want to use such libraries and are forced to explicitly add permissions. But I think that the community, as a whole, has eventually accepted this because the harm done to those who don't care is small (they just need to add the permissions), to those who do care it helps a lot, and because fewer and fewer libraries require "integrity-busting" permissions, many applications need to do absolutely nothing and get important guarantees for free.

aw1621107 13d ago
> this is very different from Rust, which requires unsafe even for things like benign write/write races, which are fairly common, and various basic data structures I know this paper [0] is quite old at this point, but the mention of benign data races reminded me of it. Would you happen to know how applicable it is to modern memory models? [0]: https://www.usenix.org/legacy/event/hotpar11/tech/final_files/Boehm.pdf https://www.usenix.org/legacy/event/hotpar11/tech/final_file...
- pron 12d ago
  Benign write/write races (when multiple threads do unordered writes of the same value to the same address) are quite common and useful, both in parallel algorithms and in lazy initialisation. Useful benign read/write races are far more rare to the point I'd say it's ok to assume they don't (or shouldn't) exist. However, in C and C++ (and Rust) benign non-atomic write/write races are UB (indeed, LLVM also treats them as potential causes of UB). In C# and in Java they are safe (although Java currently only has non-atomic writes on 32-bit machines, but soon they'll be more common when value types are enhanced). LLVM even has a specific construct to support the Java-style memory model (https://llvm.org/docs/Atomics.html#unordered https://llvm.org/docs/Atomics.html#unordered), and Zig lets you use it (https://ziglang.org/documentation/master/#atomicStore https://ziglang.org/documentation/master/#atomicStore).

aw1621107 13d ago

> In particular, you could have unsafe code in crate A, whose data are then used by crate B. Is this backwards? If B consumes data from A then to me that does not imply that A depends on anything from B; for a more concrete example that sentence reads to me like A is basically "throwing data over the wall" to B and whatever B does with said data is of no relevance to A. As a result, if B changes that shouldn't affect A. Also for what it's worth I get the impression you and treyd might be talking about slightly different things when talking about whether unsafe code composes. I believe treyd is referring to the RustBelt series of papers [0, 1], for which the statement "unsafe code composes" means (at a high level) that adding a module with a memory-safe API to a memory-safe system will result in a memory-safe system as long as the implementation upholds the safe semantics. Yes, the last bit can be a rather significant caveat, as you said. What you're talking about seems more along the lines of needing to look beyond the boundaries of unsafe blocks to prove that the unsafe block upholds its invariants, which is also true. I think you only need to check within whatever safe encapsulation boundary is relevant, though, rather than globally. [0]: https://people.mpi-sws.org/~dreyer/papers/rustbelt/paper.pdf https://people.mpi-sws.org/~dreyer/papers/rustbelt/paper.pdf [1]: https://plv.mpi-sws.org/rustbelt/rbrlx/paper.pdf https://plv.mpi-sws.org/rustbelt/rbrlx/paper.pdf

toast0 12d ago
> Is this backwards? If B consumes data from A then to me that does not imply that A depends on anything from B; for a more concrete example that sentence reads to me like A is basically "throwing data over the wall" to B and whatever B does with said data is of no relevance to A. As a result, if B changes that shouldn't affect A. This is a specifically crafted bad idea, but you could have module A use unsafe to craft a Vec<u8> that is safe to use to read or write, but not to grow or shrink. You declare an invariant that the receiver shalt not grow or shrink the Vec. If B only reads and write, you're good. But if a future B breaks the invariant, bad things happen. As I said, specifically a bad idea; there's a much better type to use if the thing can't grow or shrink... No real world example, because I don't think we've run into memory safety issues with unsafe in the Rust code base I work in... but we only use unsafe where it's required (syscalls and other FFI).