12 ms·

I wonder how it can auto generate safe bindings for C++. To do that, it would essentially need to literally prove that the C++ code is thread and memory safe,

I wonder how it can auto generate safe bindings for C++. To do that, it would essentially need to literally prove that the C++ code is thread and memory safe, which is an open research problem at Best, and probably impossible since it requires solving the halting problem. If it can do that, then the actual binding generation would be the most uninteresting part of this work.

Ygg2 6y ago
It assumes using your library is safe, same way you assume Rust std is safe, even though it uses unsafe in the background. Basically, burden of proof of using unsafe correctly is on the programmer.
- simias 6y ago
  But there's no concept of safety, lifetime and borrows in C++ (there's aliasing, but the rules are different). There's a lot of code that one could write that's perfectly kosher in C++ but a no-no in Rust. A function like memmove would be completely unsafe in Rust for instance but C++ could expose such an interface without having to do anything special.
  - rumanator 6y ago
    > There's a lot of code that one could write that's perfectly kosher in C++ but a no-no in Rust. Not a problem. Rust checks are applied to the Rust code you write or import, and everything else is just ingested as is.
  - Ygg2 6y ago
    So, Rust is to blame for C++ problems? No. C++ programs are safe under certain conditions. If they aren't then they need to be changed in C++ or in Rust interfacing code. Even if something is unsafe, it can still be used safely.
    - fluffything 6y ago
      Sure, but safe Rust functions can _always_ be used safely. If your function cannot always be used safely, either it is an `unsafe` Rust function, or your program is broken according to the Rust spec. Publishing safe Rust APIs that are _intentionally_ unsound should at best be warned about to any other crate that depends on it. Ideally, those authors and their crate would simply be banned.
      - Ygg2 6y ago
        And I explained. Either you change the C++ lib to not behave unsafely for inputs. Or you manually change the binding to keep the invariant. The assumption behind linking to C++ code is that it doesn't contain unsoundness. If it does the library was fucked way before.
        fluffything 6y ago
        I agree with you. But this discussion is about a tool that: * does not change the C++ code to behave correctly for all inputs * does not (and technically cannot) generate safe Rust bindings that keep the invariant (the bindings are automatically generated under the assumption that all C++ code is safe for all inputs).
        the_mitsuhiko 6y ago
        It seems it’s quite an absolutist way to look at safety. It establishes that nothing is allowed to ask a user to proof safety unless the unsafe keyword is used somewhere. I think this is already invalidated by countless code generation and build scripts.
        Jweb_Guru 6y ago
        The use of the unsafe keyword is how Rust's "absolutist" view is made practical... the obligation to write an extra seven to nine characters is pretty minimal compared to the burden of an actual proof. "Some automated build script probably don't respect that" isn't a great argument against that standard, I think, especially with the push to sandbox build scripts and procedural macros showing that people are still quite concerned about safety in those areas (I know you're referring to the code they generate and not the scripts themselves, but I see these as related issues of trust).
  - deleted 6y ago
    [deleted]
yoshuaw 6y ago
I read "safely call" as "does not require `unsafe {}` to call". Invariants will still need to be manually upheld to ensure the C++ code will work as expected.
- simias 6y ago
  The problem with that is that making a function safe means that you guarantee that these invariants are enforced. If you tag a function as safe erroneously you basically throw Rust's safety out of the window. Given that C++ has no explicit concept of safety it seems like it would be hard to do that automatically as soon as pointers or references are involved. As a quick example: if you have the following signature in C++: int *foo(int &a); How you automatically generate safe FFI for it? As a human doing the same task I'd have to ask myself at the very least: - Can the return value be NULL? - Can the return value alias a? - What's the lifetime of the returned value and who owns it?
  - rumanator 6y ago
    > If you tag a function as safe erroneously you basically throw Rust's safety out of the window. You really don't. Rust checks are applied to the code written in Rust. If a developer intentionally marked a dependency as safe then you get exactly what you've asked for. Sometimes it's unquestionably better to have a working system than not having one just because a pedantic compiler complains about stuff that you can't do nothing about.
    - ChrisSD 6y ago
      If an interface can break safe Rust (without that being an implementation bug) then it should not be marked as safe. Breaking safety isn't about the compiler being pedantic, it can break your real world program. Yes this may mean you sometimes can't create a safe interface to a particular foreign function. However all this means is that calling the function requires an explicit unsafe block and extra care. But that 'unsafe' marker is valuable! It's something that screams "here be dragons".
    - simias 6y ago
awestroke 6y ago
The bindings are safe, not the C++ they bind to
- fluffything 6y ago
  If the C++ they bind to is not safe, then allowing these to be called from safe Rust is unsound.
  - alvarelle 6y ago
    The point is that the C++ code should be safe because the C++ programmer should not introduce UB on its C++ code. If the C++ code invoke UB, that is a bug in the C++ code which should be found by reviewing the C++ code alone. No need to write 'unsafe' because .cpp files are already known to need carefull review.
    - masklinn 6y ago
      > The point is that the C++ code should be safe because the C++ programmer should not introduce UB on its C++ code. That's a misunderstanding of safety, and ub, and `unsafe`. The C++ code could be unsafe when called with certain values which it is not normally called with. This is common. This is also not allowed in Rust, it'd be unsound. Furthermore C++ has different notions of safety than Rust. C++ allows dangling and null pointers (whether raw or smart), it doesn't allow calling them. Rust does not allow dangling or null pointers unless they're raw. You can have a null unique_ptr, you can not have an empty Box.
      - alvarelle 6y ago
        I believe I understand correctly UB and unsafe. The cxx crate and the autocxx tool should make sure that the exposed C++ functions only take arguments types which have well defined semantics. In your example, a rust Box<T> maps to a rust::Box<T> in C++, which cannot be null. And a unique_ptr from C++ maps to a cxx::UniquePtr in rust which can be empty. If somehow the C++ code puts a dangling or null pointer into a rust::Box, that is clearly a bug in the C++ code.
steveklabnik 6y ago
See https://github.com/dtolnay/cxx/issues/1 https://github.com/dtolnay/cxx/issues/1 for some of this debate.
- fluffything 6y ago
  Ralf Jung's views there are perfectly reasonable. unsafe { ... } is a soundness proof, it reads "the code within this block is sound". People lying about having proved safety in their crates accidentally is bad. People doing this intentionally is extremely bad and I wish there was a way to automatically reject being able to depend on intentionally unsound crates in crates.io (or ban their authors are their crates from pushing anything to crates.io since they cannot be trusted). Code generators that automatically generate thousands of broken soundness proof en masse and by design are IMO the ultimate evil. They completely defeat Rust's purpose. It makes absolutely no sense to interface Rust and C++ in this way, and the people doing this would be better off by just sticking to C++ instead of trying to make safe Rust unsound. If safe Rust cannot be trusted, Rust value proposition is _dead_ (you cannot hack without feat anymore, refactor without fear, avoid segfaults, ...). This people are writing tools to automatically generate massive amounts of broken Rust code. If crates.io does not protect Rust users from them, we need a different crate repository that does.
  - throwfaraway12 6y ago
    crates.io isn't the ultimate ivory tower. It is an open space for anyone wanting to share and upload code. It is your responsibility to vet your dependencies. For anything serious, you better setup a layered review process. If you want a "vetted crates.io", then propose that. I’d be in favor. I certainly never liked crates.io to be the next NPM. But telling people their crates are "evil" and trying to get them banned is breaking the CoC. The people uploading "broken" code, no matter how much they upload, aren’t the ones breaking it.
  - steveklabnik 6y ago
    I actually disagree with you completely, but I understand where it’s coming from. I think this debate is really, really interesting.
appleflaxen 6y ago
It might just be me, but I feel like from this comment on down, everyone is saying the same thing in different words. (which is great, when the topic is a little complicated like this)
geofft 6y ago
Do you think that e.g. the Rust bindings to libgit2 or OpenSSL or libc also need to prove that the entire C library being bound is 100% thread and memory safe and free of bugs in order to expose safe wrappers?
- masklinn 6y ago
  Kinda? It needs to ensure that whatever preconditions those libraries have which are not reflected in their API because the languages they use don't allow for it are never broken. So let's say a libgit function takes a pointer (for an array) and an index, the rust bindings must ensure that the pointer is valid and the index is within the array. Will there be bugs and things which will be missed? Likely, after all we've seen that in pure-rust unsafe code, including the standard library. But the library "can't" just yolo and expose the entire thing as-is through a safe interface. As in technically it can do that just fine, but that's completely unsound even if it's effectively never called incorrectly.
  - geofft 6y ago
    So, if you restrict yourself to looking just at the API and not the implementation, it seems to me that if your library operates entirely on non-pointer C++ types (e.g., it takes in a std::string and returns an std::string), a program could automatically determine that and call the binding "safe". Would that be enough? I agree that the program should not automatically generate "safe" bindings that take pointers, because in Rust, creating and passing around raw pointers is safe but actually using them is safe. But if, let's say, you have an API that consists entirely of integers, bools, std::strings, and structs and classes thereof, what additional things would you need to check to be confident calling the binding safe? (Sure, there are weird cases here like "this function takes a long, casts it to a pointer, and dereferences it," but I assume those are uncommon enough that you'll see if you're about to create or use auto-generated bindings to such a function. I suppose there could be "this function takes an std::string and an index, and the index must be less than the length or it's UB" - are those common enough that they make this endeavor questionable?)
    - 6y ago

Ygg2 6y ago

It assumes using your library is safe, same way you assume Rust std is safe, even though it uses unsafe in the background. Basically, burden of proof of using unsafe correctly is on the programmer.

simias 6y ago
But there's no concept of safety, lifetime and borrows in C++ (there's aliasing, but the rules are different). There's a lot of code that one could write that's perfectly kosher in C++ but a no-no in Rust. A function like memmove would be completely unsafe in Rust for instance but C++ could expose such an interface without having to do anything special.
- rumanator 6y ago
  > There's a lot of code that one could write that's perfectly kosher in C++ but a no-no in Rust. Not a problem. Rust checks are applied to the Rust code you write or import, and everything else is just ingested as is.
- Ygg2 6y ago
  So, Rust is to blame for C++ problems? No. C++ programs are safe under certain conditions. If they aren't then they need to be changed in C++ or in Rust interfacing code. Even if something is unsafe, it can still be used safely.
  - fluffything 6y ago
    Sure, but safe Rust functions can _always_ be used safely. If your function cannot always be used safely, either it is an `unsafe` Rust function, or your program is broken according to the Rust spec. Publishing safe Rust APIs that are _intentionally_ unsound should at best be warned about to any other crate that depends on it. Ideally, those authors and their crate would simply be banned.
    - Ygg2 6y ago
      And I explained. Either you change the C++ lib to not behave unsafely for inputs. Or you manually change the binding to keep the invariant. The assumption behind linking to C++ code is that it doesn't contain unsoundness. If it does the library was fucked way before.
      - fluffything 6y ago
        I agree with you. But this discussion is about a tool that: * does not change the C++ code to behave correctly for all inputs * does not (and technically cannot) generate safe Rust bindings that keep the invariant (the bindings are automatically generated under the assumption that all C++ code is safe for all inputs).
        the_mitsuhiko 6y ago
        It seems it’s quite an absolutist way to look at safety. It establishes that nothing is allowed to ask a user to proof safety unless the unsafe keyword is used somewhere. I think this is already invalidated by countless code generation and build scripts.
        Jweb_Guru 6y ago
        The use of the unsafe keyword is how Rust's "absolutist" view is made practical... the obligation to write an extra seven to nine characters is pretty minimal compared to the burden of an actual proof. "Some automated build script probably don't respect that" isn't a great argument against that standard, I think, especially with the push to sandbox build scripts and procedural macros showing that people are still quite concerned about safety in those areas (I know you're referring to the code they generate and not the scripts themselves, but I see these as related issues of trust).
- deleted 6y ago
  [deleted]

yoshuaw 6y ago

I read "safely call" as "does not require `unsafe {}` to call". Invariants will still need to be manually upheld to ensure the C++ code will work as expected.

simias 6y ago
The problem with that is that making a function safe means that you guarantee that these invariants are enforced. If you tag a function as safe erroneously you basically throw Rust's safety out of the window. Given that C++ has no explicit concept of safety it seems like it would be hard to do that automatically as soon as pointers or references are involved. As a quick example: if you have the following signature in C++: int *foo(int &a); How you automatically generate safe FFI for it? As a human doing the same task I'd have to ask myself at the very least: - Can the return value be NULL? - Can the return value alias a? - What's the lifetime of the returned value and who owns it?
- rumanator 6y ago
  > If you tag a function as safe erroneously you basically throw Rust's safety out of the window. You really don't. Rust checks are applied to the code written in Rust. If a developer intentionally marked a dependency as safe then you get exactly what you've asked for. Sometimes it's unquestionably better to have a working system than not having one just because a pedantic compiler complains about stuff that you can't do nothing about.
  - ChrisSD 6y ago
    If an interface can break safe Rust (without that being an implementation bug) then it should not be marked as safe. Breaking safety isn't about the compiler being pedantic, it can break your real world program. Yes this may mean you sometimes can't create a safe interface to a particular foreign function. However all this means is that calling the function requires an explicit unsafe block and extra care. But that 'unsafe' marker is valuable! It's something that screams "here be dragons".
  - simias 6y ago

awestroke 6y ago

The bindings are safe, not the C++ they bind to

fluffything 6y ago
If the C++ they bind to is not safe, then allowing these to be called from safe Rust is unsound.
- alvarelle 6y ago
  The point is that the C++ code should be safe because the C++ programmer should not introduce UB on its C++ code. If the C++ code invoke UB, that is a bug in the C++ code which should be found by reviewing the C++ code alone. No need to write 'unsafe' because .cpp files are already known to need carefull review.
  - masklinn 6y ago
    > The point is that the C++ code should be safe because the C++ programmer should not introduce UB on its C++ code. That's a misunderstanding of safety, and ub, and `unsafe`. The C++ code could be unsafe when called with certain values which it is not normally called with. This is common. This is also not allowed in Rust, it'd be unsound. Furthermore C++ has different notions of safety than Rust. C++ allows dangling and null pointers (whether raw or smart), it doesn't allow calling them. Rust does not allow dangling or null pointers unless they're raw. You can have a null unique_ptr, you can not have an empty Box.
    - alvarelle 6y ago
      I believe I understand correctly UB and unsafe. The cxx crate and the autocxx tool should make sure that the exposed C++ functions only take arguments types which have well defined semantics. In your example, a rust Box<T> maps to a rust::Box<T> in C++, which cannot be null. And a unique_ptr from C++ maps to a cxx::UniquePtr in rust which can be empty. If somehow the C++ code puts a dangling or null pointer into a rust::Box, that is clearly a bug in the C++ code.

steveklabnik 6y ago

See https://github.com/dtolnay/cxx/issues/1 https://github.com/dtolnay/cxx/issues/1 for some of this debate.

fluffything 6y ago
Ralf Jung's views there are perfectly reasonable. unsafe { ... } is a soundness proof, it reads "the code within this block is sound". People lying about having proved safety in their crates accidentally is bad. People doing this intentionally is extremely bad and I wish there was a way to automatically reject being able to depend on intentionally unsound crates in crates.io (or ban their authors are their crates from pushing anything to crates.io since they cannot be trusted). Code generators that automatically generate thousands of broken soundness proof en masse and by design are IMO the ultimate evil. They completely defeat Rust's purpose. It makes absolutely no sense to interface Rust and C++ in this way, and the people doing this would be better off by just sticking to C++ instead of trying to make safe Rust unsound. If safe Rust cannot be trusted, Rust value proposition is _dead_ (you cannot hack without feat anymore, refactor without fear, avoid segfaults, ...). This people are writing tools to automatically generate massive amounts of broken Rust code. If crates.io does not protect Rust users from them, we need a different crate repository that does.
- throwfaraway12 6y ago
  crates.io isn't the ultimate ivory tower. It is an open space for anyone wanting to share and upload code. It is your responsibility to vet your dependencies. For anything serious, you better setup a layered review process. If you want a "vetted crates.io", then propose that. I’d be in favor. I certainly never liked crates.io to be the next NPM. But telling people their crates are "evil" and trying to get them banned is breaking the CoC. The people uploading "broken" code, no matter how much they upload, aren’t the ones breaking it.
- steveklabnik 6y ago
  I actually disagree with you completely, but I understand where it’s coming from. I think this debate is really, really interesting.

appleflaxen 6y ago

It might just be me, but I feel like from this comment on down, everyone is saying the same thing in different words. (which is great, when the topic is a little complicated like this)

geofft 6y ago

Do you think that e.g. the Rust bindings to libgit2 or OpenSSL or libc also need to prove that the entire C library being bound is 100% thread and memory safe and free of bugs in order to expose safe wrappers?

masklinn 6y ago
Kinda? It needs to ensure that whatever preconditions those libraries have which are not reflected in their API because the languages they use don't allow for it are never broken. So let's say a libgit function takes a pointer (for an array) and an index, the rust bindings must ensure that the pointer is valid and the index is within the array. Will there be bugs and things which will be missed? Likely, after all we've seen that in pure-rust unsafe code, including the standard library. But the library "can't" just yolo and expose the entire thing as-is through a safe interface. As in technically it can do that just fine, but that's completely unsound even if it's effectively never called incorrectly.
- geofft 6y ago
  So, if you restrict yourself to looking just at the API and not the implementation, it seems to me that if your library operates entirely on non-pointer C++ types (e.g., it takes in a std::string and returns an std::string), a program could automatically determine that and call the binding "safe". Would that be enough? I agree that the program should not automatically generate "safe" bindings that take pointers, because in Rust, creating and passing around raw pointers is safe but actually using them is safe. But if, let's say, you have an API that consists entirely of integers, bools, std::strings, and structs and classes thereof, what additional things would you need to check to be confident calling the binding safe? (Sure, there are weird cases here like "this function takes a long, casts it to a pointer, and dereferences it," but I assume those are uncommon enough that you'll see if you're about to create or use auto-generated bindings to such a function. I suppose there could be "this function takes an std::string and an index, and the index must be less than the length or it's UB" - are those common enough that they make this endeavor questionable?)
  - 6y ago