5 ms·

> The major difficulty here is that in general, unsafe pieces of code cannot be safely composed, even if the unsafe pieces of code are individually safe. This a

> The major difficulty here is that in general, unsafe pieces of code cannot be safely composed, even if the unsafe pieces of code are individually safe. This allows you to bypass runtime safety checks without unsafe code just by composing "safe" modules that internally use unsafe code in their implementation. This comment suggests you don't have much domain knowledge about how `unsafe` in Rust works, so I'm surprised you speak with such confidence. Your comment is flatly wrong: users using only safe code are not responsibility for guaranteeing the composed safety of the components they use (whether or not they are implemented with unsafe code). Interfaces marked safe must uphold Rust's safety guarantees, or they are incorrect. They are just wrong if they have additional untyped invariants that need to be maintained to guarantee their safety; interfaces like this must be marked `unsafe`. Because they cannot depend on untyped invariants, any correct implementation with a safe interface can be composed with any other. This ability to create safe abstractions over unsafe code which extend the reasoning ability of the type system is a fundamental value proposition of Rust.

wickawic 9y ago
Correct me if I’m wrong but I thing GP was stating that composing two “unsafe” blocks together (both of which are manually verified to work well) might interfere with each other when run simultaneously.
- tatterdemalion 9y ago
  Its possible that I misread the comment; it seemed to state that this problem extended into safe Rust, which it definitely does not.
  - madez 9y ago
    Think of two libraries that use unsafe Rust and interact with the same hardware, but work correctly when used on their own. A program written only in pure not-unsafe Rust might use these two libraries in a way that breaks because the assertions the programmers of the libaries had, like for example having exclusive access to the hardware, are wrong now. One could argue the pure not-unsafe Rust program is wrong, not the libraries. I think klodolph's comment is very thoughtful and shows a good deal of experience and domain knowledge.
    - tatterdemalion 9y ago
      There is a conflation happening here. What is the nature of this bug when you compose these two libraries together? If it is a violation of Rust's safety guarantees, then at least one of those libraries has a bug, it is exposes a safe abstraction which is not actually safe. One could not argue that the safe Rust program is wrong; the library exposing an unsafe interface as safe is unarguably wrong. If the library just behaves incorrectly in a manner disconnected from the type system because some global state was changed in a way it doesn't expect ("the hardware" in this case), then that's a normal bug & it is not connected to unsafe code at all.
      - klodolph 9y ago
        > then at least one of those libraries has a bug Yes, we agree about this point. However, the process for determining if these bugs exist is not well understood. That's what I mean when I say that this is not only a pedagogical problem--even Rust experts struggle to prove that a library containing "unsafe" blocks is safe, and more research into the area is needed.
        tatterdemalion 9y ago
        My apologies if I misunderstood you - I read your comment as suggesting that safe abstractions are "leaky" and therefore create additional responsibilities for users to validate that they are using them safely when composing them together. This is not the case unless those abstractions are incorrect - which is the same situation you are with any language, just most languages those abstractions exist within the language runtime & not in libraries.
- adwhit 9y ago
  'unsafe' doesn't mean unsafe. Unsafe means "I can't convince the compiler that this is safe. But in my context, it is." If there is any way in which a function containing an `unsafe` block may be used unsafely (specifically, violating memory-safety), then that function must also be marked as unsafe.
  - Retra 9y ago
    That's what it means if you use it properly. If you write bad code, it means "this code will break everything and the compiler won't protect you." An `unsafe` block does nothing to guarantee that you're doing something safe, which is what you seem to be saying, even if it's not what you mean to say.
    - stouset 9y ago
      This is pretty much tautological, and nobody's arguing this point. However, you have the benefit of being able to narrow your search scope to the parts of your code marked `unsafe` instead of the entire project. Most things don't need unsafe code. For the things that do, you must yourself uphold the invariant that all requirements of safety are being obeyed when transitioning out of an unsafe block. If you don't do this, bad things can happen. Other languages don't have this because they either don't offer Rust's safety guarantees in the first place, or the only way to circumvent them is to write code in C.
      - klodolph 9y ago
        > However, you have the benefit of being able to narrow your search scope to the parts of your code marked `unsafe` instead of the entire project. I may be missing some context, but this is certainly not true in Rust. In order to understand whether an individual piece of code marked `unsafe` is actually correct, you need to examine the context in which it is run and in general you could have to examine a large section of "safe" code in order to figure out whether the "unsafe" block is correct. Usually you will have to examine the entire module.
        9y ago
klodolph 9y ago
> This comment suggests you don't have much domain knowledge about how `unsafe` in Rust works, so I'm surprised you speak with such confidence. I hate being tone police, but jeez, we're having a discussion about Rust here and talking about my personal competency is inappropriate and unwelcome. The problem I'm talking about happens when you write libraries that contain "unsafe" blocks. You want to prove (or at least assure yourself) that no unsafe behavior is observable by clients of the library. However, the way to do this is not entirely clear, although there is research being done in this area. One known trap is that it is not sufficient to demonstrate that Rust code without "unsafe" blocks cannot observe unsafe behavior in your library. See: https://plv.mpi-sws.org/rustbelt/popl18/paper.pdf https://plv.mpi-sws.org/rustbelt/popl18/paper.pdf These concerns are not hypothetical, there have been soundness problems in the Rust standard library before and I expect it to happen again.
- tatterdemalion 9y ago
  Proving the correctness of unsafe code is totally different from what you talked about, which was composing different abstractions with unsafe internals together. Users of safe Rust do not need to worry about whether the composition of two safe interfaces that use unsafe internally is safe unless one of those interfaces is incorrect. Your comment would suggest that users need to think about the untyped invariants of each library they use, but this is not correct, libraries are not allowed to rely on untyped invariants for the correctness of their safe APIs.
  - klodolph 9y ago
    The problem with talking about this subject is that "safe" and "unsafe" are overloaded terms in Rust, so I can understand why you think I was talking about something different. Let R be arbitrary Rust code with no "unsafe" blocks. Let X and Y be libraries with "unsafe" blocks. You can prove that R + X is safe, and prove that R + Y is safe, but you haven't yet proven R + X + Y is safe. This is the hard part, because without an understanding of what property of X and Y individually makes R + X + Y + Z + ... safe, we don't have a good definition for what makes an interface "safe". And this is what I mean when I say that this is not only a pedagogical problem.

wickawic 9y ago

Correct me if I’m wrong but I thing GP was stating that composing two “unsafe” blocks together (both of which are manually verified to work well) might interfere with each other when run simultaneously.

tatterdemalion 9y ago
Its possible that I misread the comment; it seemed to state that this problem extended into safe Rust, which it definitely does not.
- madez 9y ago
  Think of two libraries that use unsafe Rust and interact with the same hardware, but work correctly when used on their own. A program written only in pure not-unsafe Rust might use these two libraries in a way that breaks because the assertions the programmers of the libaries had, like for example having exclusive access to the hardware, are wrong now. One could argue the pure not-unsafe Rust program is wrong, not the libraries. I think klodolph's comment is very thoughtful and shows a good deal of experience and domain knowledge.
  - tatterdemalion 9y ago
    There is a conflation happening here. What is the nature of this bug when you compose these two libraries together? If it is a violation of Rust's safety guarantees, then at least one of those libraries has a bug, it is exposes a safe abstraction which is not actually safe. One could not argue that the safe Rust program is wrong; the library exposing an unsafe interface as safe is unarguably wrong. If the library just behaves incorrectly in a manner disconnected from the type system because some global state was changed in a way it doesn't expect ("the hardware" in this case), then that's a normal bug & it is not connected to unsafe code at all.
    - klodolph 9y ago
      > then at least one of those libraries has a bug Yes, we agree about this point. However, the process for determining if these bugs exist is not well understood. That's what I mean when I say that this is not only a pedagogical problem--even Rust experts struggle to prove that a library containing "unsafe" blocks is safe, and more research into the area is needed.
      - tatterdemalion 9y ago
        My apologies if I misunderstood you - I read your comment as suggesting that safe abstractions are "leaky" and therefore create additional responsibilities for users to validate that they are using them safely when composing them together. This is not the case unless those abstractions are incorrect - which is the same situation you are with any language, just most languages those abstractions exist within the language runtime & not in libraries.
adwhit 9y ago
'unsafe' doesn't mean unsafe. Unsafe means "I can't convince the compiler that this is safe. But in my context, it is." If there is any way in which a function containing an `unsafe` block may be used unsafely (specifically, violating memory-safety), then that function must also be marked as unsafe.
- Retra 9y ago
  That's what it means if you use it properly. If you write bad code, it means "this code will break everything and the compiler won't protect you." An `unsafe` block does nothing to guarantee that you're doing something safe, which is what you seem to be saying, even if it's not what you mean to say.
  - stouset 9y ago
    This is pretty much tautological, and nobody's arguing this point. However, you have the benefit of being able to narrow your search scope to the parts of your code marked `unsafe` instead of the entire project. Most things don't need unsafe code. For the things that do, you must yourself uphold the invariant that all requirements of safety are being obeyed when transitioning out of an unsafe block. If you don't do this, bad things can happen. Other languages don't have this because they either don't offer Rust's safety guarantees in the first place, or the only way to circumvent them is to write code in C.
    - klodolph 9y ago
      > However, you have the benefit of being able to narrow your search scope to the parts of your code marked `unsafe` instead of the entire project. I may be missing some context, but this is certainly not true in Rust. In order to understand whether an individual piece of code marked `unsafe` is actually correct, you need to examine the context in which it is run and in general you could have to examine a large section of "safe" code in order to figure out whether the "unsafe" block is correct. Usually you will have to examine the entire module.
      - 9y ago

klodolph 9y ago

> This comment suggests you don't have much domain knowledge about how `unsafe` in Rust works, so I'm surprised you speak with such confidence. I hate being tone police, but jeez, we're having a discussion about Rust here and talking about my personal competency is inappropriate and unwelcome. The problem I'm talking about happens when you write libraries that contain "unsafe" blocks. You want to prove (or at least assure yourself) that no unsafe behavior is observable by clients of the library. However, the way to do this is not entirely clear, although there is research being done in this area. One known trap is that it is not sufficient to demonstrate that Rust code without "unsafe" blocks cannot observe unsafe behavior in your library. See: https://plv.mpi-sws.org/rustbelt/popl18/paper.pdf https://plv.mpi-sws.org/rustbelt/popl18/paper.pdf These concerns are not hypothetical, there have been soundness problems in the Rust standard library before and I expect it to happen again.

tatterdemalion 9y ago
Proving the correctness of unsafe code is totally different from what you talked about, which was composing different abstractions with unsafe internals together. Users of safe Rust do not need to worry about whether the composition of two safe interfaces that use unsafe internally is safe unless one of those interfaces is incorrect. Your comment would suggest that users need to think about the untyped invariants of each library they use, but this is not correct, libraries are not allowed to rely on untyped invariants for the correctness of their safe APIs.
- klodolph 9y ago
  The problem with talking about this subject is that "safe" and "unsafe" are overloaded terms in Rust, so I can understand why you think I was talking about something different. Let R be arbitrary Rust code with no "unsafe" blocks. Let X and Y be libraries with "unsafe" blocks. You can prove that R + X is safe, and prove that R + Y is safe, but you haven't yet proven R + X + Y is safe. This is the hard part, because without an understanding of what property of X and Y individually makes R + X + Y + Z + ... safe, we don't have a good definition for what makes an interface "safe". And this is what I mean when I say that this is not only a pedagogical problem.