7 ms·

I never understood that logic. You can write perfectly safe code in C where virtually everything is unsafe in the sense of the unsafe keyword. To my understan

by k__ 5y ago

I never understood that logic. You can write perfectly safe code in C where virtually everything is unsafe in the sense of the unsafe keyword. To my understanding, the idea of unsafe is that you now leave the guard rails that Rust provides and have to think for yourself.

setpatchaddress 5y ago
Excellent -- so your large C codebase is free of security bugs, something no software vendor has managed to accomplish. Ritchie and Thompson themselves didn't manage that feat. Tell us how you do it, please, so the world may learn.
- throwaway_c2 5y ago
  Oh, that’s actually simple. Enforcing the “malloc once” rule gets you like halfway there, and using custom hardened code for common stuff like string manipulation finishes the job. Things like that are why some C codebases are large in the first place - you have to reinvent your own safe space from scratch. The funny thing is that it’s actually harder to protect from certain classes of bugs in Rust. For example, you cannot uncouple from the global allocator as easy as you can in C/C++, and if you do, you do it with “unsafe” code. That doesn’t make Rust a bad language, it’s just that you can see some of the inherent flaws only if you had written safe C code previously and then you find out that some basic techniques don’t translate.
kibwen 5y ago
The idea of the `unsafe` keyword in Rust isn't that a single use of it anywhere in the dependency chain must make the entire code base suspect. Instead, the point is that encapsulation can be used to create primitives that rely on `unsafe` internally but whose interface doesn't provide any way to violate memory safety; downstream consumers of the interface shouldn't need to worry themselves about upholding memory safety. Here's a contrived example, imagine a library with the following public function: pub fn index_or_zero_i_guess_lol_idk(array: Vec<u8>, index: usize) -> u8 { if index < array.len() { // SAFETY: array index must be in bounds unsafe { *array.get_unchecked(index) } } else { 0 // yolo } } Does a downstream consumer of this library need to do anything to ensure memory safety of their own code? No, because the unsafe `get_unchecked` function has exactly one caller-upheld safety invariant, that the array index is in bounds (documented at https://doc.rust-lang.org/std/primitive.slice.html#safety https://doc.rust-lang.org/std/primitive.slice.html#safety ), and because this code makes sure to uphold the invariant, the buck stops here. A downstream consumer need not use `unsafe` at all, and thus not give up the guard rails. And even in a single codebase, `unsafe` only "infects" the module that it's in, not the whole codebase, since a module is the unit of encapsulation in Rust. If you make your modules containing `unsafe` as small as possible, then you reduce amount of code that needs audited for safety. And that's the whole point of `unsafe` in Rust: to reduce the scope of code that can potentially be memory-unsafe. It firmly places responsibility on users of `unsafe` to ensure that they uphold the necessary safety invariants; there's never any question as to which code is responsible for memory unsafety, as it can always be traced back to a single use of the `unsafe` keyword somewhere that has failed to uphold a safety invariant. It doesn't outright eliminate bugs--humans are fallible--but it makes it obvious when they might happen, it makes it easier to have confidence that they won't happen, and when they do happen it's easier to figure out why.
- incrudible 5y ago
  A downstream consumer might not need to use the keyword unsafe, but by calling such a function, they have de facto lost memory safety. It is a nice-to-have to know that only the unsafe blocks of code are able to violate memory safety, but once memory is corrupted, errors can manifest anywhere. Rust in practice is likely not as safe an ecosystem as C#, because Rust users are more likely to use unsafe blocks for performance tricks. In my view, Rust should have a safe keyword to mark functions that are transitively not unsafe, much like the pure keyword in some languages marks transitive functional purity.
  - lucian1900 5y ago
    P/Invoke is not uncommon either, especially in Windows. Rust and C# offer similar guarantees.
  - eloff 5y ago
    > A downstream consumer might not need to use the keyword unsafe, but by calling such a function, they have de facto lost memory safety. That's a silly definition of memory safety. You can't write a non trivial rust program that doesn't use unsafe in it's dependencies. No interior mutability, no allocation, no containers. The point is these uses of unsafe are checked by the author to ensure they can't introduce unsound behavior through the exposed safe API. > Rust in practice is likely not as safe an ecosystem as C#, because Rust users are more likely to use unsafe blocks for performance tricks. Sort of agree with you here. Unsafe code is rare in C#. But C# has race conditions so some kinds of bugs are more common. But C# isn't as suited for systems programming type of problems or really high performance code.
    - incrudible 5y ago
      It is an honest definition of memory safety. The fact that you can not write a non trivial Rust program without unsafe dependencies is not an argument against that definition. Most languages considered memory safe have some sort virtual machine model where non trivial programs can be written without any use of unsafe. The virtual machine implementation itself may be unsafe of course, but that is a pretty solid boundary. In Rust, that boundary is easily crossed by anyone, for any reason, to the point where it sips into 50% of dependencies, as this paper shows. I am not saying that C# evades this issue in principle, obviously it has the same functionality, it is just not used as liberally. Concurrency is orthogonal.
tialaramex 5y ago
Unsafe doesn't remove all the guard rails, but it allows you to do a handful (five) of tremendously unsafe things, for which you then take all responsibility as the compiler cannot analyse them for safety. * You can dereference raw pointers. Are they pointing at anything at all? At something that type pointer might reasonably point to? Not the compiler's job to check, if you screwed up your program now has Undefined Behaviour like C or C++ * You can access C-style unions. This one says it might be a boolean, a pointer to something, or an integer. You decided to guess it's... a boolean. If you're wrong you get Undefined Behaviour. * You can mutate (change) static variables. Global cheese flavour is Cheddar? Let's change it to Gorgonzola, no wait Parmesan. Was anybody else using that for anything? No idea, too bad, you might introduce Undefined Behaviour if you didn't think this through. * You can call functions Rust labelled "unsafe". These functions come with instructions about rules you must follow to use them safely. If you violate any of those instructions, all bets are off, but if you obey the instructions whoever wrote the function (which may be the Rust standard library) promises that was safe. * You can implement special Traits labelled "unsafe" to implement. These Traits, unlike most Rust traits, have to promise they're correct. If you claim to implement Searcher, a Trait for text processing, and you get it wrong, other people's code may blow up. Whereas I have types like Funhouse<> which claim to implement the trait Eq (promising they understand how equality works) and then as a goof they utterly refuse to do so correctly, that's a safe trait, nothing blows up. My types don't work in a sensible way (Funhouses are simultaneously equal to and not equal to every other Funhouse including themselves, which is stupid), but your program doesn't have Undefined Behaviour if you use these types, they're just annoying. But, lots of things that would cause Undefined Behaviour in some popular languages are impossible even in unsafe Rust. Suppose I have a function that takes an array of 1024 bytes. I try to do { let z = array[1040]; } -- That won't compile, this array isn't big enough. If I write unsafe { let z = array[1040]; } it still doesn't compile, didn't I get it, this array isn't big enough. OK, let's have the function take a parameter k, and at runtime we can set k to 1040 and try that way. If I write { let z = array[k]; } when k is 1040 the program panics. The array isn't big enough. If I write unsafe { let z = array[k]; } it still panics and the compiler even warns me that unsafe isn't doing anything for me here and I should remove it. Similarly although Rust provides checked arithmetic (ie you can explicitly say "Tell me the answer to x + 100, or, if that would overflow, tell me it didn't work") its default integer arithmetic will panic in debug builds if you overflow and, if you missed that in debug, but it happens in production, you get overflow, which may very well not be what you wanted, but you apparently didn't even know it could happen, so, good luck with that. Either way, no Undefined Behaviour. 255_u8 + 255_u8 is a compile error, a runtime panic or at worst it's 254, but it definitely is not zero, forty-two, or unrelated program misbehaviour.
pornel 5y ago
The difference is that in Rust you can build safe (foolproof) abstractions on top unsafe code. OTOH, in C you can't write a library that returns a pointer and prevents users from misusing it (e.g. ensure they can't use it after your lib frees the data, or improperly share it between threads). In C, the risk of unsafety and misuse of APIs remains a problem for every downstream user. For example, Rust's String type internally allocates, reallocates, and frees memory. It can have partially uninitialized buffer, and lots of other "unsafe" things. But there's no way to cause UB when using safe Rust's String. You can't cause use-after-free. You can't cause a data race. You can't overflow the buffer. You can't even break its UTF-8 encoding. That's because String's public interface only exposes safe methods. The unsafe String internals were written and checked for correctness, and from then on they're safe to use for everyone else.
- kristianp 5y ago
  Aren't they saying though, that there could be a bug in the string unsafe code that causes UBs. You can't guarantee that (although this example is unlikely).
  - jhanschoo 5y ago
    That is true in general. But 1. unsafe code in the standard library is of course more battle-tested than ad-hoc unsafe code, and 2. there are efforts to formally verify Rust. http://plv.mpi-sws.org/rustbelt/ http://plv.mpi-sws.org/rustbelt/ and https://internals.rust-lang.org/t/announcing-the-formal-verification-working-group/7240 https://internals.rust-lang.org/t/announcing-the-formal-veri...
  - littlestymaar 5y ago
    There could, and these kind of bugs have happened a few times already, but fortunately they are few and far between, and as the sibling comment mentioned there's some work in progress to formal prove the std lib, and because the std lib is much smaller than the amount of code that uses the stdlib, it's much less work to formally verify the stdlib than it would be to verify for instance all C++ programms using the C++ stdlib. [1]: the most famous bug in the standard library exposing an unsound construction is the “leakpocalypse”: https://github.com/rust-lang/rust/issues/24292 https://github.com/rust-lang/rust/issues/24292
topspin 5y ago
"I never understood that logic." Pragmatic techniques mystify some. Some code is beyond the ability of a compiler to verify because the necessary information is missing. Rust acknowledges this reality and accommodates such code through unsafe. The presumption inherent in this is that unsafe is used only when deemed necessary. Yes, deemed; a decision made by some flawed soul, as opposed to a compiler. This is a pragmatic policy and it cannot be reconciled by the unreasonable.
mamcx 5y ago
> You can write perfectly safe code in C where virtually everything is unsafe in the sense of the unsafe keyword. The key here: Rust SURFACE AS TYPES this information. Is similar to this piece of code (imagine is python): sum(a,b) what it actually do? WHO KNOWS. But this: sum(a:int,b:int):int is informative. And the "type system" can verify its use. So "unsafe" is part of the type system, type checks that Rust use to COMMUNICATE and CHECK the use of code. So, if I, as user see this: unsafe fn raw_memory(...) I know the type system of Rust was bypassed and must TRUST the HUMAN know how correctly code what is inside. --- The major advantage here, is that the blast area of this stuff is limited and for the rest of regular code, the type system verify the code and remove the need to manually inspect (and make tests!) for a lot of trivial (and not so much1) stuff. Is a total game-changer, in special for a system language.

by k__ 5y ago

setpatchaddress 5y ago

Excellent -- so your large C codebase is free of security bugs, something no software vendor has managed to accomplish. Ritchie and Thompson themselves didn't manage that feat. Tell us how you do it, please, so the world may learn.

throwaway_c2 5y ago
Oh, that’s actually simple. Enforcing the “malloc once” rule gets you like halfway there, and using custom hardened code for common stuff like string manipulation finishes the job. Things like that are why some C codebases are large in the first place - you have to reinvent your own safe space from scratch. The funny thing is that it’s actually harder to protect from certain classes of bugs in Rust. For example, you cannot uncouple from the global allocator as easy as you can in C/C++, and if you do, you do it with “unsafe” code. That doesn’t make Rust a bad language, it’s just that you can see some of the inherent flaws only if you had written safe C code previously and then you find out that some basic techniques don’t translate.

kibwen 5y ago

The idea of the `unsafe` keyword in Rust isn't that a single use of it anywhere in the dependency chain must make the entire code base suspect. Instead, the point is that encapsulation can be used to create primitives that rely on `unsafe` internally but whose interface doesn't provide any way to violate memory safety; downstream consumers of the interface shouldn't need to worry themselves about upholding memory safety. Here's a contrived example, imagine a library with the following public function: pub fn index_or_zero_i_guess_lol_idk(array: Vec<u8>, index: usize) -> u8 { if index < array.len() { // SAFETY: array index must be in bounds unsafe { *array.get_unchecked(index) } } else { 0 // yolo } } Does a downstream consumer of this library need to do anything to ensure memory safety of their own code? No, because the unsafe `get_unchecked` function has exactly one caller-upheld safety invariant, that the array index is in bounds (documented at https://doc.rust-lang.org/std/primitive.slice.html#safety https://doc.rust-lang.org/std/primitive.slice.html#safety ), and because this code makes sure to uphold the invariant, the buck stops here. A downstream consumer need not use `unsafe` at all, and thus not give up the guard rails. And even in a single codebase, `unsafe` only "infects" the module that it's in, not the whole codebase, since a module is the unit of encapsulation in Rust. If you make your modules containing `unsafe` as small as possible, then you reduce amount of code that needs audited for safety. And that's the whole point of `unsafe` in Rust: to reduce the scope of code that can potentially be memory-unsafe. It firmly places responsibility on users of `unsafe` to ensure that they uphold the necessary safety invariants; there's never any question as to which code is responsible for memory unsafety, as it can always be traced back to a single use of the `unsafe` keyword somewhere that has failed to uphold a safety invariant. It doesn't outright eliminate bugs--humans are fallible--but it makes it obvious when they might happen, it makes it easier to have confidence that they won't happen, and when they do happen it's easier to figure out why.

incrudible 5y ago
A downstream consumer might not need to use the keyword unsafe, but by calling such a function, they have de facto lost memory safety. It is a nice-to-have to know that only the unsafe blocks of code are able to violate memory safety, but once memory is corrupted, errors can manifest anywhere. Rust in practice is likely not as safe an ecosystem as C#, because Rust users are more likely to use unsafe blocks for performance tricks. In my view, Rust should have a safe keyword to mark functions that are transitively not unsafe, much like the pure keyword in some languages marks transitive functional purity.
- lucian1900 5y ago
  P/Invoke is not uncommon either, especially in Windows. Rust and C# offer similar guarantees.
- eloff 5y ago
  > A downstream consumer might not need to use the keyword unsafe, but by calling such a function, they have de facto lost memory safety. That's a silly definition of memory safety. You can't write a non trivial rust program that doesn't use unsafe in it's dependencies. No interior mutability, no allocation, no containers. The point is these uses of unsafe are checked by the author to ensure they can't introduce unsound behavior through the exposed safe API. > Rust in practice is likely not as safe an ecosystem as C#, because Rust users are more likely to use unsafe blocks for performance tricks. Sort of agree with you here. Unsafe code is rare in C#. But C# has race conditions so some kinds of bugs are more common. But C# isn't as suited for systems programming type of problems or really high performance code.
  - incrudible 5y ago
    It is an honest definition of memory safety. The fact that you can not write a non trivial Rust program without unsafe dependencies is not an argument against that definition. Most languages considered memory safe have some sort virtual machine model where non trivial programs can be written without any use of unsafe. The virtual machine implementation itself may be unsafe of course, but that is a pretty solid boundary. In Rust, that boundary is easily crossed by anyone, for any reason, to the point where it sips into 50% of dependencies, as this paper shows. I am not saying that C# evades this issue in principle, obviously it has the same functionality, it is just not used as liberally. Concurrency is orthogonal.

tialaramex 5y ago

Unsafe doesn't remove all the guard rails, but it allows you to do a handful (five) of tremendously unsafe things, for which you then take all responsibility as the compiler cannot analyse them for safety. * You can dereference raw pointers. Are they pointing at anything at all? At something that type pointer might reasonably point to? Not the compiler's job to check, if you screwed up your program now has Undefined Behaviour like C or C++ * You can access C-style unions. This one says it might be a boolean, a pointer to something, or an integer. You decided to guess it's... a boolean. If you're wrong you get Undefined Behaviour. * You can mutate (change) static variables. Global cheese flavour is Cheddar? Let's change it to Gorgonzola, no wait Parmesan. Was anybody else using that for anything? No idea, too bad, you might introduce Undefined Behaviour if you didn't think this through. * You can call functions Rust labelled "unsafe". These functions come with instructions about rules you must follow to use them safely. If you violate any of those instructions, all bets are off, but if you obey the instructions whoever wrote the function (which may be the Rust standard library) promises that was safe. * You can implement special Traits labelled "unsafe" to implement. These Traits, unlike most Rust traits, have to promise they're correct. If you claim to implement Searcher, a Trait for text processing, and you get it wrong, other people's code may blow up. Whereas I have types like Funhouse<> which claim to implement the trait Eq (promising they understand how equality works) and then as a goof they utterly refuse to do so correctly, that's a safe trait, nothing blows up. My types don't work in a sensible way (Funhouses are simultaneously equal to and not equal to every other Funhouse including themselves, which is stupid), but your program doesn't have Undefined Behaviour if you use these types, they're just annoying. But, lots of things that would cause Undefined Behaviour in some popular languages are impossible even in unsafe Rust. Suppose I have a function that takes an array of 1024 bytes. I try to do { let z = array[1040]; } -- That won't compile, this array isn't big enough. If I write unsafe { let z = array[1040]; } it still doesn't compile, didn't I get it, this array isn't big enough. OK, let's have the function take a parameter k, and at runtime we can set k to 1040 and try that way. If I write { let z = array[k]; } when k is 1040 the program panics. The array isn't big enough. If I write unsafe { let z = array[k]; } it still panics and the compiler even warns me that unsafe isn't doing anything for me here and I should remove it. Similarly although Rust provides checked arithmetic (ie you can explicitly say "Tell me the answer to x + 100, or, if that would overflow, tell me it didn't work") its default integer arithmetic will panic in debug builds if you overflow and, if you missed that in debug, but it happens in production, you get overflow, which may very well not be what you wanted, but you apparently didn't even know it could happen, so, good luck with that. Either way, no Undefined Behaviour. 255_u8 + 255_u8 is a compile error, a runtime panic or at worst it's 254, but it definitely is not zero, forty-two, or unrelated program misbehaviour.

pornel 5y ago

The difference is that in Rust you can build safe (foolproof) abstractions on top unsafe code. OTOH, in C you can't write a library that returns a pointer and prevents users from misusing it (e.g. ensure they can't use it after your lib frees the data, or improperly share it between threads). In C, the risk of unsafety and misuse of APIs remains a problem for every downstream user. For example, Rust's String type internally allocates, reallocates, and frees memory. It can have partially uninitialized buffer, and lots of other "unsafe" things. But there's no way to cause UB when using safe Rust's String. You can't cause use-after-free. You can't cause a data race. You can't overflow the buffer. You can't even break its UTF-8 encoding. That's because String's public interface only exposes safe methods. The unsafe String internals were written and checked for correctness, and from then on they're safe to use for everyone else.

kristianp 5y ago
Aren't they saying though, that there could be a bug in the string unsafe code that causes UBs. You can't guarantee that (although this example is unlikely).
- jhanschoo 5y ago
  That is true in general. But 1. unsafe code in the standard library is of course more battle-tested than ad-hoc unsafe code, and 2. there are efforts to formally verify Rust. http://plv.mpi-sws.org/rustbelt/ http://plv.mpi-sws.org/rustbelt/ and https://internals.rust-lang.org/t/announcing-the-formal-verification-working-group/7240 https://internals.rust-lang.org/t/announcing-the-formal-veri...
- littlestymaar 5y ago
  There could, and these kind of bugs have happened a few times already, but fortunately they are few and far between, and as the sibling comment mentioned there's some work in progress to formal prove the std lib, and because the std lib is much smaller than the amount of code that uses the stdlib, it's much less work to formally verify the stdlib than it would be to verify for instance all C++ programms using the C++ stdlib. [1]: the most famous bug in the standard library exposing an unsound construction is the “leakpocalypse”: https://github.com/rust-lang/rust/issues/24292 https://github.com/rust-lang/rust/issues/24292

topspin 5y ago

"I never understood that logic." Pragmatic techniques mystify some. Some code is beyond the ability of a compiler to verify because the necessary information is missing. Rust acknowledges this reality and accommodates such code through unsafe. The presumption inherent in this is that unsafe is used only when deemed necessary. Yes, deemed; a decision made by some flawed soul, as opposed to a compiler. This is a pragmatic policy and it cannot be reconciled by the unreasonable.

mamcx 5y ago

> You can write perfectly safe code in C where virtually everything is unsafe in the sense of the unsafe keyword. The key here: Rust SURFACE AS TYPES this information. Is similar to this piece of code (imagine is python): sum(a,b) what it actually do? WHO KNOWS. But this: sum(a:int,b:int):int is informative. And the "type system" can verify its use. So "unsafe" is part of the type system, type checks that Rust use to COMMUNICATE and CHECK the use of code. So, if I, as user see this: unsafe fn raw_memory(...) I know the type system of Rust was bypassed and must TRUST the HUMAN know how correctly code what is inside. --- The major advantage here, is that the blast area of this stuff is limited and for the rest of regular code, the type system verify the code and remove the need to manually inspect (and make tests!) for a lot of trivial (and not so much1) stuff. Is a total game-changer, in special for a system language.