5 ms·

That isn't really true. "unsafe" specifically means "memory-unsafe". The checking is left to the humans, and so is the justification, but the meaning is not. S

by remram 5y ago

That isn't really true. "unsafe" specifically means "memory-unsafe". The checking is left to the humans, and so is the justification, but the meaning is not. See https://doc.rust-lang.org/book/ch19-01-unsafe-rust.html https://doc.rust-lang.org/book/ch19-01-unsafe-rust.html Check out also the "Rust considers it safe to" section of https://doc.rust-lang.org/nomicon/what-unsafe-does.html https://doc.rust-lang.org/nomicon/what-unsafe-does.html

josephg 5y ago
I feel like there's plenty of instances where unsafe is used in rust to indicate library level UB, without actually being memory-unsafe. (Like, where using a method in a certain way may violate a runtime assumption of a library, causing application level UB). For example, String::from_utf8_unchecked[1]. There's a comment in the documentation trying to justify why invalid strings could cause memory unsafety, but its pretty weak: > This function is unsafe because it does not check that the bytes passed to it are valid UTF-8. If this constraint is violated, it may cause memory unsafety issues with future users of the String, as the rest of the standard library assumes that Strings are valid UTF-8. Like, sure, we could make that argument for any runtime invariant. "Look, this is totally memory safe but if you call this method we assume this constraint is valid. We're marking this unsafe anyway because we want people to take care in using it. So uh, its memory-unsafe because violating this constraint could cause memory unsafety issues in the future or something? Yeah that'll do." It seems like lots of folks quietly want unsafe to mean "this method skips runtime checks", with no specific grounding in memory-unsafety. And the line between those two ideas is super blurry in practice, even in the standard library. [1] https://doc.rust-lang.org/std/string/struct.String.html#method.from_utf8_unchecked https://doc.rust-lang.org/std/string/struct.String.html#meth...
- deleted 5y ago
  [deleted]
- Dylan16807 5y ago
  Ensuring that values actually are a specific type isn't directly memory safety, but if you don't do it then you can't even have an exhaustive match without the danger of your program going wild. I see enforcing utf8 as about on par with enforcing that a boolean be 0 or 1, and not 5. How do you feel about this list of what's unsafe and undefined? https://doc.rust-lang.org/nomicon/what-unsafe-does.html https://doc.rust-lang.org/nomicon/what-unsafe-does.html
  - josephg 5y ago
    > I see enforcing utf8 as about on par with enforcing that a boolean be 0 or 1, and not 5. UTF8 correctness is an invariant of the code. Its not something the compiler understands or cares about it. But really, I get it! My point is that rust is rife with people marking functions "unsafe" when they mean a method will cause bugs / "runtime UB" if used improperly. Even the standard library does this, so it feels a bit rich to cry foul when 3rd party libraries (ab)use "unsafe" in the same way. I certainly use unsafe like that, in order to mark methods which, if misused, will violate internal invariants of my code. How does that list of unsafe features justify the unsafe marker on String::from_utf8_unchecked?
    - adgjlsfhk1 5y ago
      the key thing here is that if you know that no safe code contains invalid utf8, it is safe to do unsafe indexing into tables that are only sized to deal with valid utf8. as such, this works in practice lead to ub in otherwise correct code, so it should be marked unsafe.
    - Dylan16807 5y ago
      Well how do you feel about the comparison to booleans? If I had a way to set some booleans "unchecked", in a way that could make the numerical value be 5, would you label it unsafe? I really think it should be unsafe, because it's crazy to make ifs or switches on booleans be non-exhaustive, and it's also crazy to have an "invalid boolean" path anywhere one is used. When you have an encapsulated value, with special accessors to make sure it's always in range, then you don't have to have an unchecked setter function at all. But if you do add one, I think it makes sense to make it unsafe. That function is basically allowing a blind memcpy over a piece of data. It ties into safety pretty strongly. That kind of access is a lot like dereferencing a raw pointer, and not making it unsafe means you allow a lot of those "invalid value" cases in the link to be possible in safe code.
      - josephg 5y ago
        I agree. I think "unchecked" booleans which could contain 5 should be unsafe too. I'd be happy to extend the definition of "unsafe" to mean "If you misuse this, you may violate some internal invariants of the library. And that may cause unexpected bugs." Thats a broader, but in my opinion much more clear definition of unsafe, which covers how the keyword is actually used in actual code. (In std::String and elsewhere). I've implemented high performance b-tree and skiplist implementations in rust. There's plenty of functions in both libraries which I've marked as unsafe because if you use them carelessly, you'll violate some internal invariants. Will the library break as a result? Yes. Will the resulting bugs include memory corruption? I really have no idea, and I don't really care enough to go in and test that. So I've marked the methods unsafe, and provided safe APIs for consumers to use instead. Do you think this is an appropriate use for unsafe? If not, I'm curious to hear why.
- zozbot234 5y ago
  > Like, sure, we could make that argument for any runtime invariant. One difference is that Rust has a separate type for String-like objects that do not respect the invariant, namely Vec<u8>. So those who wish to write memory-safe code acting on arbitrary bytes can simply use that. Also, it's quite normal for an invariant defined entirely in Safe Rust to impact memory safety in a very real sense; consider Send and Sync. These are seemingly arbitrary labels, but their semantics is nonetheless quite well defined.
- aliceryhl 5y ago
  It is perfectly reasonable that `from_utf8_unchecked` is unsafe because the sequence of operations `from_utf8_unchecked(array).chars().next()` will call [1] and can trigger memory unsafety with the `unwrap_unchecked` call if the array is not valid utf-8. This means that at least one of the three functions must be marked unsafe. [1] https://github.com/rust-lang/rust/blob/027a232755fa9728e9699337267f6675dfd0a8ba/library/core/src/str/validations.rs#L36-L70 https://github.com/rust-lang/rust/blob/027a232755fa9728e9699...
- deleted 5y ago
  [deleted]

by remram 5y ago

josephg 5y ago

I feel like there's plenty of instances where unsafe is used in rust to indicate library level UB, without actually being memory-unsafe. (Like, where using a method in a certain way may violate a runtime assumption of a library, causing application level UB). For example, String::from_utf8_unchecked[1]. There's a comment in the documentation trying to justify why invalid strings could cause memory unsafety, but its pretty weak: > This function is unsafe because it does not check that the bytes passed to it are valid UTF-8. If this constraint is violated, it may cause memory unsafety issues with future users of the String, as the rest of the standard library assumes that Strings are valid UTF-8. Like, sure, we could make that argument for any runtime invariant. "Look, this is totally memory safe but if you call this method we assume this constraint is valid. We're marking this unsafe anyway because we want people to take care in using it. So uh, its memory-unsafe because violating this constraint could cause memory unsafety issues in the future or something? Yeah that'll do." It seems like lots of folks quietly want unsafe to mean "this method skips runtime checks", with no specific grounding in memory-unsafety. And the line between those two ideas is super blurry in practice, even in the standard library. [1] https://doc.rust-lang.org/std/string/struct.String.html#method.from_utf8_unchecked https://doc.rust-lang.org/std/string/struct.String.html#meth...

deleted 5y ago
[deleted]
Dylan16807 5y ago
Ensuring that values actually are a specific type isn't directly memory safety, but if you don't do it then you can't even have an exhaustive match without the danger of your program going wild. I see enforcing utf8 as about on par with enforcing that a boolean be 0 or 1, and not 5. How do you feel about this list of what's unsafe and undefined? https://doc.rust-lang.org/nomicon/what-unsafe-does.html https://doc.rust-lang.org/nomicon/what-unsafe-does.html
- josephg 5y ago
  > I see enforcing utf8 as about on par with enforcing that a boolean be 0 or 1, and not 5. UTF8 correctness is an invariant of the code. Its not something the compiler understands or cares about it. But really, I get it! My point is that rust is rife with people marking functions "unsafe" when they mean a method will cause bugs / "runtime UB" if used improperly. Even the standard library does this, so it feels a bit rich to cry foul when 3rd party libraries (ab)use "unsafe" in the same way. I certainly use unsafe like that, in order to mark methods which, if misused, will violate internal invariants of my code. How does that list of unsafe features justify the unsafe marker on String::from_utf8_unchecked?
  - adgjlsfhk1 5y ago
    the key thing here is that if you know that no safe code contains invalid utf8, it is safe to do unsafe indexing into tables that are only sized to deal with valid utf8. as such, this works in practice lead to ub in otherwise correct code, so it should be marked unsafe.
  - Dylan16807 5y ago
    Well how do you feel about the comparison to booleans? If I had a way to set some booleans "unchecked", in a way that could make the numerical value be 5, would you label it unsafe? I really think it should be unsafe, because it's crazy to make ifs or switches on booleans be non-exhaustive, and it's also crazy to have an "invalid boolean" path anywhere one is used. When you have an encapsulated value, with special accessors to make sure it's always in range, then you don't have to have an unchecked setter function at all. But if you do add one, I think it makes sense to make it unsafe. That function is basically allowing a blind memcpy over a piece of data. It ties into safety pretty strongly. That kind of access is a lot like dereferencing a raw pointer, and not making it unsafe means you allow a lot of those "invalid value" cases in the link to be possible in safe code.
    - josephg 5y ago
      I agree. I think "unchecked" booleans which could contain 5 should be unsafe too. I'd be happy to extend the definition of "unsafe" to mean "If you misuse this, you may violate some internal invariants of the library. And that may cause unexpected bugs." Thats a broader, but in my opinion much more clear definition of unsafe, which covers how the keyword is actually used in actual code. (In std::String and elsewhere). I've implemented high performance b-tree and skiplist implementations in rust. There's plenty of functions in both libraries which I've marked as unsafe because if you use them carelessly, you'll violate some internal invariants. Will the library break as a result? Yes. Will the resulting bugs include memory corruption? I really have no idea, and I don't really care enough to go in and test that. So I've marked the methods unsafe, and provided safe APIs for consumers to use instead. Do you think this is an appropriate use for unsafe? If not, I'm curious to hear why.
zozbot234 5y ago
> Like, sure, we could make that argument for any runtime invariant. One difference is that Rust has a separate type for String-like objects that do not respect the invariant, namely Vec<u8>. So those who wish to write memory-safe code acting on arbitrary bytes can simply use that. Also, it's quite normal for an invariant defined entirely in Safe Rust to impact memory safety in a very real sense; consider Send and Sync. These are seemingly arbitrary labels, but their semantics is nonetheless quite well defined.
aliceryhl 5y ago
It is perfectly reasonable that `from_utf8_unchecked` is unsafe because the sequence of operations `from_utf8_unchecked(array).chars().next()` will call [1] and can trigger memory unsafety with the `unwrap_unchecked` call if the array is not valid utf-8. This means that at least one of the three functions must be marked unsafe. [1] https://github.com/rust-lang/rust/blob/027a232755fa9728e9699337267f6675dfd0a8ba/library/core/src/str/validations.rs#L36-L70 https://github.com/rust-lang/rust/blob/027a232755fa9728e9699...
deleted 5y ago
[deleted]