15 ms·

Ralf Jung's views there are perfectly reasonable. unsafe { ... } is a soundness proof, it reads "the code within this block is sound". People lying abo

Ralf Jung's views there are perfectly reasonable. unsafe { ... } is a soundness proof, it reads "the code within this block is sound". People lying about having proved safety in their crates accidentally is bad. People doing this intentionally is extremely bad and I wish there was a way to automatically reject being able to depend on intentionally unsound crates in crates.io (or ban their authors are their crates from pushing anything to crates.io since they cannot be trusted). Code generators that automatically generate thousands of broken soundness proof en masse and by design are IMO the ultimate evil. They completely defeat Rust's purpose. It makes absolutely no sense to interface Rust and C++ in this way, and the people doing this would be better off by just sticking to C++ instead of trying to make safe Rust unsound. If safe Rust cannot be trusted, Rust value proposition is _dead_ (you cannot hack without feat anymore, refactor without fear, avoid segfaults, ...). This people are writing tools to automatically generate massive amounts of broken Rust code. If crates.io does not protect Rust users from them, we need a different crate repository that does.

throwfaraway12 6y ago
crates.io isn't the ultimate ivory tower. It is an open space for anyone wanting to share and upload code. It is your responsibility to vet your dependencies. For anything serious, you better setup a layered review process. If you want a "vetted crates.io", then propose that. I’d be in favor. I certainly never liked crates.io to be the next NPM. But telling people their crates are "evil" and trying to get them banned is breaking the CoC. The people uploading "broken" code, no matter how much they upload, aren’t the ones breaking it.
steveklabnik 6y ago
I actually disagree with you completely, but I understand where it’s coming from. I think this debate is really, really interesting.
- dtolnay 6y ago
  I've been planning on the cxx side to update my attribute macro to require `unsafe mod ffi {...}` or `unsafe extern "C" {...}` to signal presence of a proof obligation. I think that resolves all concerns from the January discussion, including viewpoints I don't necessarily agree with, without sacrificing any ease of use. Just haven't found time to make the compiler PR to allow exposing that syntax to macros yet, but I will.
  - fluffything 6y ago
    That would completely resolve my concern about cxx. (i still think its a soundness bug in Rust that `extern { }` does not require unsafe since one can use it to trigger UB in safe Rust code)
    - tick_tock_tick 6y ago
      What's your thoughts on #[no_mangle]? https://github.com/rust-lang/rust/issues/28179 https://github.com/rust-lang/rust/issues/28179 Frankly Rust values being a usable language too much to be 100% sound.
      - Jweb_Guru 6y ago
        I don't understand why you and others think that bringing up existing and irritating soundness holes in Rust is an argument for introducing new ones. Whether or not you think CXX exposes a sound API, it's a bad argument. This one is literally a bug that we can hopefully fix at some point, just like the floating point casts one was finally fixed.
      - fluffything 6y ago
        #[no_mangle], like many other Rust features, is unsound; that's a fact and not an opinion, there is a bug in the tracker open for it and labelled I-unsound (accepted as an unsound bug). It is a low priority issue, but will be fixed eventually. > Frankly Rust values being a usable language too much to be 100% sound. Rust has a really good track record of identifying, prioritizing, and fixing soundness bugs (e.g. it took 4 years of work to fix one floating-point soundness bug!). Many people continuously work on this at the academic, toolchain, and backend (llvm, crane lift) levels. There is also a lot of people continuously working on making sure that new language features like async/await, const generics, specialization, GATs, ... are sound. TBH i'm surprised to learn that not all Rust core members consider soundness to be a Rust core value. Maybe it isn't a Rust core value? (it is for me; without it, Rust makes no sense as a language to me)
  - steveklabnik 6y ago
- fluffything 6y ago
  We already had this debate many times. One of the last ones was the actix debacle, which was contained and accidental and ended up badly. This is IMO infinitely worse.
  - steveklabnik 6y ago
    I think they are categorically different debates.
    - fluffything 6y ago
      To me they are categorically identical. Rust code author knowingly exposes unsound safe Rust APIs. The details are different. Actix did so in their own project and were willing to fix it (the debacle was mostly social, and in some sense even accidental), while here exposing unsound safe Rust APIs is the whole raison d'etre of the project. AutoCXX doesn't even check that the underlying C++ code does not change. So even if an autogenerated safe Rust API happens to be "accidentally safe", that can change any time without Rust users knowing.
      - steveklabnik 6y ago
        > Rust code author knowingly exposes unsound safe Rust APIs. This is where we disagree. I 100% agree that knowingly exposing an unsound API to safe Rust is a bad thing. However, what I see is the user of cxx/autocxx saying "I am asserting that this API is sound." > AutoCXX doesn't even check that the underlying C++ code does not change This is no different than any other Rust code calling into C++ code. The author has to declare that they believe it is safe either way.
        fluffything 6y ago
        > However, what I see is the user of cxx/autocxx saying "I am asserting that this API is sound." And since the API of this library allows this assertion to be performed in safe Rust code, when it fails, safe Rust code has UB, which according to the Rust language reference makes the API of this crate unsound. Unsound safe Rust APIs are broken Rust code. The Rust language spec and toolchain make no guarantees about what the behavior of these APIs is. So that's what I see here. Just another safe Rust API that is broken by design. Maybe with the twist that this crate is actually a factory to generates thousands of those broken safe Rust APIs in masse, which kind of makes it worse than your usual "broken API" bug (which is what these are, these are soundness bugs in Rust libraries).
- Jweb_Guru 6y ago
  I don't actually understand your position on this argument at all... maybe you can clarify? I guess it doesn't really matter since it was resolved in a way that satisfies everyone, it just makes me think I'm missing something. If I grep a dependency for `unsafe`, don't find any, but I have UB in my application from that library, I am going to be very, very unhappy.
  - steveklabnik 6y ago
    I wrote it up as a blog post: https://steveklabnik.com/blog/the-cxx-debate https://steveklabnik.com/blog/the-cxx-debate TL;DR: I think this is just regular old composition of Rust features, and that using the library itself is the proof obligation. I can see why others don't like it. I think dtolnay has come to a good compromise.
    - Jweb_Guru 6y ago
      All of the examples in your blog post are examples of code that is obviously safe. I think extending that analogy to something as large as a nontrivial C++ library (one you'd need CXX to integrate with) is such a huge stretch that it kind of feels disingenuous as an argument. However, your current stated justification ("using the library is equivalent to unsafe") is at least a little more understandable to me; nonetheless, the standard in Rust is syntactically writing `unsafe` and I'm glad that dtolnay found a compromise that worked for everyone. (I will point out, however, that I still seriously doubt that every single one of these autogenerated functions is safe, and will probably avoid libraries that use CXX for that reason. But at least now someone who hasn't heard of CXX can come to that conclusion independently :)).
      - steveklabnik 6y ago
        Well, I do that partially because I think that it's the core of the issue here. Saying "hey I think this makes it too easy to make mistakes" is a very different argument than "obviously this library is inherently unsound."
- simias 6y ago
  I'm really surprised to read this, it's so obvious to me that what fluffything says is correct that I can't quite imagine what the counter-argument looks like. I kinda feel like the original sin in this discussion is that "unsafe {}" should really have been called "safe {}" or "sound {}" since it's really the programmer telling the compiler "hey, this block is safe to run as-is, I've checked, trust me on that". An automatic C++ binding library is in no position to make this assertion. It's weird to quote the Holy Scripture back at you of all people but here it goes (emphasis mine): https://doc.rust-lang.org/beta/reference/behavior-considered-undefined.html https://doc.rust-lang.org/beta/reference/behavior-considered... > Rust code is incorrect if it exhibits any of the behaviors in the following list. This includes code within unsafe blocks and unsafe functions. unsafe only means that avoiding undefined behavior is on the programmer; it does not change anything about the fact that Rust programs must never cause undefined behavior. > It is the programmer's responsibility when writing unsafe code to ensure that any safe code interacting with the unsafe code cannot trigger these behaviors. unsafe code that satisfies this property for any safe client is called sound; if unsafe code can be misused by safe code to exhibit undefined behavior, it is unsound. This library generates safe bindings to unsafe FFI calls without actually checking that said code is safe to call that way. It's therefore totally unsound until proven otherwise. So what's the argument here? That the C++ code might be safe to call that way if we're lucky? I mean, sure, but how is that reasonable or practical? I definitely don't want a FFI library to tag functions as safe for me before I had the time to validate it, it seems to easy to let something bad slip unnoticed. If the interface is unsafe it means that I need to review it and manually mark it as safe once I've taken due diligence and I can vouch that the interface is actually sound. And I definitely don't want a third-party crate to expose a safe-but-unsound Rust interface with a big disclaimer that "basically it's just raw calls to C++, make sure you use it right otherwise it's segfault time!". Again, it's mind boggling to me that this is controversial. It's not just bullshit language lawyering either, I think it's a terrible idea both in theory and in practice. I can already see the Rust crates leaking unsound """safe""" interfaces left and right because they're just a thin automatically-generated wrapper around C++ header files.
  - steveklabnik 6y ago
    I wrote up a blog post about it, I linked to it in a sibling comment, I don't want to spam :) > So what's the argument here? That the C++ code might be safe to call that way if we're lucky? No, the argument is that we've checked that the C++ code is safe to call, just like any other FFI. We've reduced the boilerplate of doing so with a macro, just like any other boilerplate might be. > I definitely don't want a FFI library to tag functions as safe for me before I had the time to validate it, Do you validate every single unsafe code in every single library you use? Even the standard library? Every time? > And I definitely don't want a third-party crate to expose a safe-but-unsound Rust interface with a big disclaimer that "basically it's just raw calls to C++, make sure you use it right otherwise it's segfault time!". I do not either, but CXX doesn't change this in any meaningful way. This could still happen before.
kibwen 6y ago
Even if someone uploads Rust bindings to C or C++ code to crates.io, as a library user you would still need to manually install the C/C++ library on your end, along with a compiler. I agree that we need to have strong social customs to enforce the meaning of safety where the Rust compiler cannot, but when it comes to FFI-induced unsafety I don't think we don't need to worry about library authors sneaking random unsafely-bound C or C++ libs into the dependencies of a crates.io lib without the user's realization.
- est31 6y ago
  1. this only makes interfacing with C++ libraries safe, not with C or the C-ish subset of C++. So no C pointers without unsafe. 2. many C/C++ library bindings in Rust ship the C code with them, so no need to install the library, only the compiler.
- fluffything 6y ago
  > Even if someone uploads Rust bindings to C or C++ code to crates.io, as a library user you would still need to manually install the C/C++ library on your end, along with a compiler. Not if the library comes with your system. One can use this macro to generate a more "ergonomic" "safe libc" crate, that just introduces UB in safe Rust.
pjmlp 6y ago
Which is why going back to Burroughs, Modula variants, Oberon variants and even .NET, all offer the mechanism to say no linking of unsafe code.
Matthias247 6y ago
So I read about things being the ultimate evil. But what is your counter proposal to make the world a better place? Rewrite everything in Rust in one go? Not going to happen. The last teams I worked on all maintained a giant pile of C and C++ code. Which obviously sometimes is buggy, and sometimes might even exhibit memory safety issues. I’m looking for every opportunity to make things better and using Rust for to improve safety and reliability. However due to realities this isn’t going to happen at once. So now we have 2 choices: Option 1: use bindings and replace things step by step, knowing that we have still unsafe code (but less so!) Option 2: Stick with C and C++ I think everyone agrees that option 1 is better than 2. But reading about it as „the ultimate evil“ makes me sound very uncomfortable.
- The_rationalist 6y ago
  I think everyone agrees that option 1 is better than 2. What? It seems pretty reasonable to be against it. Code clarity is the number one priority and the ffi are going to add cognitive overhead / clutter / boilerplate which is harmful.
- fluffything 6y ago
  > Rewrite everything in Rust in one go? No? Just do what every good Rust programmer does today instead? If I have a C++ function like this: // If a == nullptr the behavior is undefined void foo(int* a); the right way to provide a safe Rust API for it is to write: mod ffi { extern "C" { fn foo(a: *mut c_int); } } fn foo(a: *mut c_int) { assert!(!a.is_null()); unsafe { ffi::foo(a) } } The crates being discussed here generate: mod ffi { extern "C" { fn foo(a: *mut c_int); } } fn foo(a: *mut c_int) { unsafe { ffi::foo(a) } } instead (notice the missing assert), which is broken Rust code according to the Rust spec, because now `foo` will introduce undefined behavior into safe Rust every time its safely called with a null pointer. That is, doing this makes _all_ Rust safe code "unsafe", making the unsafe Rust keyword essentially meaningless, and negating any advantage that Rust has over C++ (memory safety, thread safety, lacks of segfaults, refactoring without introducing errors, etc.). So you end up with 2 languages in your codebase, + a lot of glue boilerplate, for very little win. If this is what you want, you are better off just sticking with C++ instead. What Firefox, Servo, and any other good C++ project that cannot/does not want to review all C++ APIs does is to just write: extern "C" { fn foo(a: *mut c_int); } instead. That's less code, and it is correct code. When a programmer needs to call foo, they need to write "unsafe { foo(a) }", and that often leads to them actually checking "foo"'s API docs, and explaining why the call is safe (maybe the assert is not needed because a cannot be null. The burden of doing the work is on the caller. Firefox and Servo heavily use `rust-bindgen`, which will automatically generate all these FFI wrappers with a correct ABIs for you. The main difference is that rust-bindgen does not attempt to falsely convey the idea that these APIs are safe to call.
Rusky 6y ago
There is no conflict between (auto)cxx and the view that `unsafe` blocks assert the soundness of their contents. Using cxx does not, in fact, remove any `unsafe` blocks- thus it is not "lying," nor is it "the ultimate evil." (Such hyperbole makes productive discussion in this space rather difficult...) Consider the (completely and utterly standard) practice of writing a safe Rust library wrapping a C++ library. To be sound, this crate has two obligations: First, its `extern {}` function signatures must match the C++ library's. Second, its public API must enforce any additional soundness invariants from the C++ library. Ensuring that signatures match is tedious and entirely mechanical- automating it increases confidence that a program is sound! Crucially, there is more to signature matching than mere arity and type layout- C++ vocabulary types express ownership information that bindgen alone does not capture! Cxx isolates all the auditing work related to C++ vocabulary types, like `std::vector` or `std::unique_ptr`. A single audit of cxx covers this aspect of all cxx-using crates. This is an enormous win if your goal is to integrate Rust with a large C++ codebase like Gecko or Chromium! The kernel of truth to your argument lies only in that second obligation. As Ralf concludes in that same issue thread, the problem of generated `unsafe` blocks is not new to cxx, but a near-universal aspect of Rust/C++ bindings. The solution is not, and never was, to give up and hand-write a bunch of `unsafe` blocks. Instead, the solution is simply to ensure that auditors can easily locate generated `unsafe` blocks and the soundness assertions they represent. A typical solution here (also mentioned in that thread) is to add "unsafe" to the macro name or input. (Dtolnay already plans to do this for cxx.) Personally, I would argue that this is not even cxx's failing, but a tooling issue- if we're going to use macros to generate bindings, then our auditing tools should see through those macros.
- fluffything 6y ago
  > Using cxx does not, in fact, remove any `unsafe` blocks- thus it is not "lying," nor is it "the ultimate evil." (Such hyperbole makes productive discussion in this space rather difficult...) If I can write an unsound Rust program using autocxx or cxx safe Rust APIs, then these APIs are unsound. Whether these APIs are macros or function calls does not matter. There is no distinction at the API level about whether incorrect unsafe code is expanded, or called into. The API is safe, and for these crates, the currently-safe API introduces undefined behavior. cxx will be fixed to require unsafe, but AFAICT autocxx usecase is to satisfy Chromiums requirements which require completely avoiding unsafe.
geofft 6y ago
Do you think that a crate that allows you to write to files should be banned from crates.io, because you can write to /proc/self/mem without writing the word "unsafe"?
- Jweb_Guru 6y ago
  I actually find that hole in Rust really bad and would be happy if Rust's runtime closed this loophole. Not sure why people think "hah! You'd have to disable being able to write bytes to random memory addresses, too!" is an argument ad absurdum, the fact that you can do that is terrible and dangerous.
  - geofft 6y ago
    How would you close that loophole, exactly? Would you prevent running a subprocess that writes to /proc/$pid/mem? Would you prevent running all subprocesses, or somehow only those that edit /proc/$pid/mem? Like, I get that you want the loophole to not exist, but does there exist a possible world in which Rust closed the loophole and you would actually be happy about it? The fact that you can write bytes to random memory addresses if you intentionally set out to do so is not what Rust's safety system is about. Rust's safety system is a tool for the programmer to build working programs. It is not an end goal in itself - hence why unsafe exists and why so much of the standard library and so many third party crates use safe. It's a tool you have to use correctly, sure, but it's only a tool. We don't avoid memory corruption and data races because they are inherently bad - we avoid them because they cause bad things to happen, like getting the wrong results out of a computation or allowing attackers to subvert program control flow. If you want to 100% guarantee that a program does not bad things, you're looking for a sandbox. Compile your code to wasm, or run it in a VM, or something, and then even if you have memory corruption or data races, it can't escape the sandbox and read the rest of your files. Or if you want to 100% guarantee that your program has no internal bugs, write it in Coq or ATS or something. Rust has a specific goal, and it does not involve being either a sandbox or a theorem prover.
    - Jweb_Guru 6y ago
nullc 6y ago
Has anyone suggested adding unsafe type bitmask? E.g. "This code and its transitive dependencies has been carefully validated to be correct", "I had to do this to make it compile", and "This is a FFI that is inherently potentially unsafe". Then the unsafty of any function would be defined as the logical OR of it and its dependencies... and you could filter what you're willing to use -- maybe with exceptions programmed for specific unsafe blocks, so you could e.g. except specific FFI interfaces without taking random ones people sneak in while they're adding malware to deep dependencies in crates.io to get silently embedded into your software.

throwfaraway12 6y ago

crates.io isn't the ultimate ivory tower. It is an open space for anyone wanting to share and upload code. It is your responsibility to vet your dependencies. For anything serious, you better setup a layered review process. If you want a "vetted crates.io", then propose that. I’d be in favor. I certainly never liked crates.io to be the next NPM. But telling people their crates are "evil" and trying to get them banned is breaking the CoC. The people uploading "broken" code, no matter how much they upload, aren’t the ones breaking it.

steveklabnik 6y ago

I actually disagree with you completely, but I understand where it’s coming from. I think this debate is really, really interesting.

dtolnay 6y ago
I've been planning on the cxx side to update my attribute macro to require `unsafe mod ffi {...}` or `unsafe extern "C" {...}` to signal presence of a proof obligation. I think that resolves all concerns from the January discussion, including viewpoints I don't necessarily agree with, without sacrificing any ease of use. Just haven't found time to make the compiler PR to allow exposing that syntax to macros yet, but I will.
- fluffything 6y ago
  That would completely resolve my concern about cxx. (i still think its a soundness bug in Rust that `extern { }` does not require unsafe since one can use it to trigger UB in safe Rust code)
  - tick_tock_tick 6y ago
    What's your thoughts on #[no_mangle]? https://github.com/rust-lang/rust/issues/28179 https://github.com/rust-lang/rust/issues/28179 Frankly Rust values being a usable language too much to be 100% sound.
    - Jweb_Guru 6y ago
      I don't understand why you and others think that bringing up existing and irritating soundness holes in Rust is an argument for introducing new ones. Whether or not you think CXX exposes a sound API, it's a bad argument. This one is literally a bug that we can hopefully fix at some point, just like the floating point casts one was finally fixed.
    - fluffything 6y ago
      #[no_mangle], like many other Rust features, is unsound; that's a fact and not an opinion, there is a bug in the tracker open for it and labelled I-unsound (accepted as an unsound bug). It is a low priority issue, but will be fixed eventually. > Frankly Rust values being a usable language too much to be 100% sound. Rust has a really good track record of identifying, prioritizing, and fixing soundness bugs (e.g. it took 4 years of work to fix one floating-point soundness bug!). Many people continuously work on this at the academic, toolchain, and backend (llvm, crane lift) levels. There is also a lot of people continuously working on making sure that new language features like async/await, const generics, specialization, GATs, ... are sound. TBH i'm surprised to learn that not all Rust core members consider soundness to be a Rust core value. Maybe it isn't a Rust core value? (it is for me; without it, Rust makes no sense as a language to me)
- steveklabnik 6y ago
fluffything 6y ago
We already had this debate many times. One of the last ones was the actix debacle, which was contained and accidental and ended up badly. This is IMO infinitely worse.
- steveklabnik 6y ago
  I think they are categorically different debates.
  - fluffything 6y ago
    To me they are categorically identical. Rust code author knowingly exposes unsound safe Rust APIs. The details are different. Actix did so in their own project and were willing to fix it (the debacle was mostly social, and in some sense even accidental), while here exposing unsound safe Rust APIs is the whole raison d'etre of the project. AutoCXX doesn't even check that the underlying C++ code does not change. So even if an autogenerated safe Rust API happens to be "accidentally safe", that can change any time without Rust users knowing.
    - steveklabnik 6y ago
      > Rust code author knowingly exposes unsound safe Rust APIs. This is where we disagree. I 100% agree that knowingly exposing an unsound API to safe Rust is a bad thing. However, what I see is the user of cxx/autocxx saying "I am asserting that this API is sound." > AutoCXX doesn't even check that the underlying C++ code does not change This is no different than any other Rust code calling into C++ code. The author has to declare that they believe it is safe either way.
      - fluffything 6y ago
        > However, what I see is the user of cxx/autocxx saying "I am asserting that this API is sound." And since the API of this library allows this assertion to be performed in safe Rust code, when it fails, safe Rust code has UB, which according to the Rust language reference makes the API of this crate unsound. Unsound safe Rust APIs are broken Rust code. The Rust language spec and toolchain make no guarantees about what the behavior of these APIs is. So that's what I see here. Just another safe Rust API that is broken by design. Maybe with the twist that this crate is actually a factory to generates thousands of those broken safe Rust APIs in masse, which kind of makes it worse than your usual "broken API" bug (which is what these are, these are soundness bugs in Rust libraries).
Jweb_Guru 6y ago
I don't actually understand your position on this argument at all... maybe you can clarify? I guess it doesn't really matter since it was resolved in a way that satisfies everyone, it just makes me think I'm missing something. If I grep a dependency for `unsafe`, don't find any, but I have UB in my application from that library, I am going to be very, very unhappy.
- steveklabnik 6y ago
  I wrote it up as a blog post: https://steveklabnik.com/blog/the-cxx-debate https://steveklabnik.com/blog/the-cxx-debate TL;DR: I think this is just regular old composition of Rust features, and that using the library itself is the proof obligation. I can see why others don't like it. I think dtolnay has come to a good compromise.
  - Jweb_Guru 6y ago
    All of the examples in your blog post are examples of code that is obviously safe. I think extending that analogy to something as large as a nontrivial C++ library (one you'd need CXX to integrate with) is such a huge stretch that it kind of feels disingenuous as an argument. However, your current stated justification ("using the library is equivalent to unsafe") is at least a little more understandable to me; nonetheless, the standard in Rust is syntactically writing `unsafe` and I'm glad that dtolnay found a compromise that worked for everyone. (I will point out, however, that I still seriously doubt that every single one of these autogenerated functions is safe, and will probably avoid libraries that use CXX for that reason. But at least now someone who hasn't heard of CXX can come to that conclusion independently :)).
    - steveklabnik 6y ago
      Well, I do that partially because I think that it's the core of the issue here. Saying "hey I think this makes it too easy to make mistakes" is a very different argument than "obviously this library is inherently unsound."
simias 6y ago
I'm really surprised to read this, it's so obvious to me that what fluffything says is correct that I can't quite imagine what the counter-argument looks like. I kinda feel like the original sin in this discussion is that "unsafe {}" should really have been called "safe {}" or "sound {}" since it's really the programmer telling the compiler "hey, this block is safe to run as-is, I've checked, trust me on that". An automatic C++ binding library is in no position to make this assertion. It's weird to quote the Holy Scripture back at you of all people but here it goes (emphasis mine): https://doc.rust-lang.org/beta/reference/behavior-considered-undefined.html https://doc.rust-lang.org/beta/reference/behavior-considered... > Rust code is incorrect if it exhibits any of the behaviors in the following list. This includes code within unsafe blocks and unsafe functions. unsafe only means that avoiding undefined behavior is on the programmer; it does not change anything about the fact that Rust programs must never cause undefined behavior. > It is the programmer's responsibility when writing unsafe code to ensure that any safe code interacting with the unsafe code cannot trigger these behaviors. unsafe code that satisfies this property for any safe client is called sound; if unsafe code can be misused by safe code to exhibit undefined behavior, it is unsound. This library generates safe bindings to unsafe FFI calls without actually checking that said code is safe to call that way. It's therefore totally unsound until proven otherwise. So what's the argument here? That the C++ code might be safe to call that way if we're lucky? I mean, sure, but how is that reasonable or practical? I definitely don't want a FFI library to tag functions as safe for me before I had the time to validate it, it seems to easy to let something bad slip unnoticed. If the interface is unsafe it means that I need to review it and manually mark it as safe once I've taken due diligence and I can vouch that the interface is actually sound. And I definitely don't want a third-party crate to expose a safe-but-unsound Rust interface with a big disclaimer that "basically it's just raw calls to C++, make sure you use it right otherwise it's segfault time!". Again, it's mind boggling to me that this is controversial. It's not just bullshit language lawyering either, I think it's a terrible idea both in theory and in practice. I can already see the Rust crates leaking unsound """safe""" interfaces left and right because they're just a thin automatically-generated wrapper around C++ header files.
- steveklabnik 6y ago
  I wrote up a blog post about it, I linked to it in a sibling comment, I don't want to spam :) > So what's the argument here? That the C++ code might be safe to call that way if we're lucky? No, the argument is that we've checked that the C++ code is safe to call, just like any other FFI. We've reduced the boilerplate of doing so with a macro, just like any other boilerplate might be. > I definitely don't want a FFI library to tag functions as safe for me before I had the time to validate it, Do you validate every single unsafe code in every single library you use? Even the standard library? Every time? > And I definitely don't want a third-party crate to expose a safe-but-unsound Rust interface with a big disclaimer that "basically it's just raw calls to C++, make sure you use it right otherwise it's segfault time!". I do not either, but CXX doesn't change this in any meaningful way. This could still happen before.

kibwen 6y ago

Even if someone uploads Rust bindings to C or C++ code to crates.io, as a library user you would still need to manually install the C/C++ library on your end, along with a compiler. I agree that we need to have strong social customs to enforce the meaning of safety where the Rust compiler cannot, but when it comes to FFI-induced unsafety I don't think we don't need to worry about library authors sneaking random unsafely-bound C or C++ libs into the dependencies of a crates.io lib without the user's realization.

est31 6y ago
1. this only makes interfacing with C++ libraries safe, not with C or the C-ish subset of C++. So no C pointers without unsafe. 2. many C/C++ library bindings in Rust ship the C code with them, so no need to install the library, only the compiler.
fluffything 6y ago
> Even if someone uploads Rust bindings to C or C++ code to crates.io, as a library user you would still need to manually install the C/C++ library on your end, along with a compiler. Not if the library comes with your system. One can use this macro to generate a more "ergonomic" "safe libc" crate, that just introduces UB in safe Rust.

pjmlp 6y ago

Which is why going back to Burroughs, Modula variants, Oberon variants and even .NET, all offer the mechanism to say no linking of unsafe code.

Matthias247 6y ago

So I read about things being the ultimate evil. But what is your counter proposal to make the world a better place? Rewrite everything in Rust in one go? Not going to happen. The last teams I worked on all maintained a giant pile of C and C++ code. Which obviously sometimes is buggy, and sometimes might even exhibit memory safety issues. I’m looking for every opportunity to make things better and using Rust for to improve safety and reliability. However due to realities this isn’t going to happen at once. So now we have 2 choices: Option 1: use bindings and replace things step by step, knowing that we have still unsafe code (but less so!) Option 2: Stick with C and C++ I think everyone agrees that option 1 is better than 2. But reading about it as „the ultimate evil“ makes me sound very uncomfortable.

The_rationalist 6y ago
I think everyone agrees that option 1 is better than 2. What? It seems pretty reasonable to be against it. Code clarity is the number one priority and the ffi are going to add cognitive overhead / clutter / boilerplate which is harmful.
fluffything 6y ago
> Rewrite everything in Rust in one go? No? Just do what every good Rust programmer does today instead? If I have a C++ function like this: // If a == nullptr the behavior is undefined void foo(int* a); the right way to provide a safe Rust API for it is to write: mod ffi { extern "C" { fn foo(a: *mut c_int); } } fn foo(a: *mut c_int) { assert!(!a.is_null()); unsafe { ffi::foo(a) } } The crates being discussed here generate: mod ffi { extern "C" { fn foo(a: *mut c_int); } } fn foo(a: *mut c_int) { unsafe { ffi::foo(a) } } instead (notice the missing assert), which is broken Rust code according to the Rust spec, because now `foo` will introduce undefined behavior into safe Rust every time its safely called with a null pointer. That is, doing this makes _all_ Rust safe code "unsafe", making the unsafe Rust keyword essentially meaningless, and negating any advantage that Rust has over C++ (memory safety, thread safety, lacks of segfaults, refactoring without introducing errors, etc.). So you end up with 2 languages in your codebase, + a lot of glue boilerplate, for very little win. If this is what you want, you are better off just sticking with C++ instead. What Firefox, Servo, and any other good C++ project that cannot/does not want to review all C++ APIs does is to just write: extern "C" { fn foo(a: *mut c_int); } instead. That's less code, and it is correct code. When a programmer needs to call foo, they need to write "unsafe { foo(a) }", and that often leads to them actually checking "foo"'s API docs, and explaining why the call is safe (maybe the assert is not needed because a cannot be null. The burden of doing the work is on the caller. Firefox and Servo heavily use `rust-bindgen`, which will automatically generate all these FFI wrappers with a correct ABIs for you. The main difference is that rust-bindgen does not attempt to falsely convey the idea that these APIs are safe to call.

Rusky 6y ago

There is no conflict between (auto)cxx and the view that `unsafe` blocks assert the soundness of their contents. Using cxx does not, in fact, remove any `unsafe` blocks- thus it is not "lying," nor is it "the ultimate evil." (Such hyperbole makes productive discussion in this space rather difficult...) Consider the (completely and utterly standard) practice of writing a safe Rust library wrapping a C++ library. To be sound, this crate has two obligations: First, its `extern {}` function signatures must match the C++ library's. Second, its public API must enforce any additional soundness invariants from the C++ library. Ensuring that signatures match is tedious and entirely mechanical- automating it increases confidence that a program is sound! Crucially, there is more to signature matching than mere arity and type layout- C++ vocabulary types express ownership information that bindgen alone does not capture! Cxx isolates all the auditing work related to C++ vocabulary types, like `std::vector` or `std::unique_ptr`. A single audit of cxx covers this aspect of all cxx-using crates. This is an enormous win if your goal is to integrate Rust with a large C++ codebase like Gecko or Chromium! The kernel of truth to your argument lies only in that second obligation. As Ralf concludes in that same issue thread, the problem of generated `unsafe` blocks is not new to cxx, but a near-universal aspect of Rust/C++ bindings. The solution is not, and never was, to give up and hand-write a bunch of `unsafe` blocks. Instead, the solution is simply to ensure that auditors can easily locate generated `unsafe` blocks and the soundness assertions they represent. A typical solution here (also mentioned in that thread) is to add "unsafe" to the macro name or input. (Dtolnay already plans to do this for cxx.) Personally, I would argue that this is not even cxx's failing, but a tooling issue- if we're going to use macros to generate bindings, then our auditing tools should see through those macros.

fluffything 6y ago
> Using cxx does not, in fact, remove any `unsafe` blocks- thus it is not "lying," nor is it "the ultimate evil." (Such hyperbole makes productive discussion in this space rather difficult...) If I can write an unsound Rust program using autocxx or cxx safe Rust APIs, then these APIs are unsound. Whether these APIs are macros or function calls does not matter. There is no distinction at the API level about whether incorrect unsafe code is expanded, or called into. The API is safe, and for these crates, the currently-safe API introduces undefined behavior. cxx will be fixed to require unsafe, but AFAICT autocxx usecase is to satisfy Chromiums requirements which require completely avoiding unsafe.

geofft 6y ago

Do you think that a crate that allows you to write to files should be banned from crates.io, because you can write to /proc/self/mem without writing the word "unsafe"?

Jweb_Guru 6y ago
I actually find that hole in Rust really bad and would be happy if Rust's runtime closed this loophole. Not sure why people think "hah! You'd have to disable being able to write bytes to random memory addresses, too!" is an argument ad absurdum, the fact that you can do that is terrible and dangerous.
- geofft 6y ago
  How would you close that loophole, exactly? Would you prevent running a subprocess that writes to /proc/$pid/mem? Would you prevent running all subprocesses, or somehow only those that edit /proc/$pid/mem? Like, I get that you want the loophole to not exist, but does there exist a possible world in which Rust closed the loophole and you would actually be happy about it? The fact that you can write bytes to random memory addresses if you intentionally set out to do so is not what Rust's safety system is about. Rust's safety system is a tool for the programmer to build working programs. It is not an end goal in itself - hence why unsafe exists and why so much of the standard library and so many third party crates use safe. It's a tool you have to use correctly, sure, but it's only a tool. We don't avoid memory corruption and data races because they are inherently bad - we avoid them because they cause bad things to happen, like getting the wrong results out of a computation or allowing attackers to subvert program control flow. If you want to 100% guarantee that a program does not bad things, you're looking for a sandbox. Compile your code to wasm, or run it in a VM, or something, and then even if you have memory corruption or data races, it can't escape the sandbox and read the rest of your files. Or if you want to 100% guarantee that your program has no internal bugs, write it in Coq or ATS or something. Rust has a specific goal, and it does not involve being either a sandbox or a theorem prover.
  - Jweb_Guru 6y ago

nullc 6y ago

Has anyone suggested adding unsafe type bitmask? E.g. "This code and its transitive dependencies has been carefully validated to be correct", "I had to do this to make it compile", and "This is a FFI that is inherently potentially unsafe". Then the unsafty of any function would be defined as the logical OR of it and its dependencies... and you could filter what you're willing to use -- maybe with exceptions programmed for specific unsafe blocks, so you could e.g. except specific FFI interfaces without taking random ones people sneak in while they're adding malware to deep dependencies in crates.io to get silently embedded into your software.