5 ms·

>Anyone willing to fund such an effort? No, not yet. The Rust ecosystem is still not mature enough. One of the big problems is Cargo which sidesteps distro pac

>Anyone willing to fund such an effort? No, not yet. The Rust ecosystem is still not mature enough. One of the big problems is Cargo which sidesteps distro packaging and pulls dependencies from some random server. This doesn't fly if you actually want reproducible builds. The other problems are the complete disregard for dynamic linking and the amount of churn in the standard library and compiler. Give Rust a few more years for things to stabilize and for people to figure out a reasonable way to handle these problems. The work will probably fall on the distro maintainers. My opinion is that if you're looking for a short-term investment that someone might want to make towards fixing these problems, try working on static analysis security tools for C and C++. It could not be easier to make these now with libclang. Alternatively, maybe get your company to fund a purchase of PVS-Studio or something like that.

jcranmer 7y ago
cargo vendor will avoid downloading dependencies from "some random server."
- cycloptic 7y ago
  Distros still won't ship that because it causes needless code duplication. Edit: I don't mean this as a general dismissal of that strategy, but more to illustrate what problems you'll run into trying to get Rust packages into a distro. A huge amount of work has gone into most distros to "un-vendor" dependencies because it leads to a lot of problems.
  - jcranmer 7y ago
    It is possible to vendor with a system-wide repository of packages to use. This is how Debian handles cargo dependencies, and I think Nix does something similar as well.
    - cycloptic 7y ago
      This only works if you keep your list of dependencies small and limited to the ones that are stable and have been tested and packaged. It still seems common to get in npm-like situations with Rust programs where typing "cargo build" installs hundreds of packages. This is what I mean by the ecosystem lacking maturity.
JoshTriplett 7y ago
Distribution maintainers have already figured out how to package Rust in a modular fashion and build without touching the network. Cargo specifically provides facilities to help distribution maintainers provide dependencies themselves. Based on those facilities, I built the initial prototype of Rust packaging for Debian ("debcargo"), and the current Debian Rust maintainers have made further huge advances in packaging. So no, Cargo doesn't "sidestep" distribution packaging, it works with distribution packaging. Also, neither the standard library nor the compiler have "churn"; we're very careful to make sure old code still compiles with current Rust.
- daurnimator 7y ago
  On arch we haven't yet. We use Cargo.lock files to get reproducible builds, but trying to get rust to follow normal practices has been difficult.
- cycloptic 7y ago
  Yes, debcargo is a great solution. Thank you for writing that. And really, I mean it, it's awesome. But my point (which you seem to be agreeing with) is that we have a while to wait while the distro people do their thing. Not just Debian. We can't just take random packages off github and put them in any distro, there is more to it than just running debcargo. >Also, neither the standard library nor the compiler have "churn"; we're very careful to make sure old code still compiles with current Rust. When did this happen? Last I checked Rust still had no LTS policy and there were no stability guarantees around the syntax, ABI, API, compiler flags, Cargo file format, etc. If some policy has changed here, please link me to any info.
  - JoshTriplett 7y ago
    > Yes, debcargo is a great solution. Thank you for writing that. And really, I mean it, it's awesome. Just to reiterate: I wrote the initial prototype, and massive credit goes to the current maintainers who picked it up and ran with it into production. > there were no stability guarantees around the syntax, ABI, API, compiler flags, Cargo file format, etc. There are absolutely stability guarantees around syntax, around compiler flags, around API, and around the Cargo file format. Code from old Rust and old Cargo will build and run with new Rust and new Cargo. We take a great deal of care to preserve that stability guarantee. The policies here have remained essentially the same since 1.0 (https://blog.rust-lang.org/2014/10/30/Stability.html https://blog.rust-lang.org/2014/10/30/Stability.html), though we've since taken further care to specify them more precisely and to stabilize further areas. (Source: I'm one of the leads of the Rust language team, and I'm on the Rust cargo team. Other Rust teams, such as the Rust library team, follow the same principles.)
unlinked_dll 7y ago
(as of iirc 1.39) Cargo supports custom registries on stable. That means you can pull your dep to whatever hosting solution you want and leave it. As well, pulling from a random server isn't the problem. Trusting a version to be a particular commit is, which crates.io handles as well as Cargo via cargo publish and the lock file. If you pull a dep, the lock file will guarantee that whatever you pull is going to be identical to what was published (assuming you didn't add a git field to the dependencies list or say version = "*"). As for churn in the std or dynamic linking, I've never had an issue using Rust in production and getting reproducible builds otherwise. Same is true for Clang or GCC/stdlib. APIs aren't being changed, binaries may shift, but unless you're on nightly you're probably not going to see anything break - just get better. But you can handle that, just don't update the compiler until you're ready.

jcranmer 7y ago

cargo vendor will avoid downloading dependencies from "some random server."

cycloptic 7y ago
Distros still won't ship that because it causes needless code duplication. Edit: I don't mean this as a general dismissal of that strategy, but more to illustrate what problems you'll run into trying to get Rust packages into a distro. A huge amount of work has gone into most distros to "un-vendor" dependencies because it leads to a lot of problems.
- jcranmer 7y ago
  It is possible to vendor with a system-wide repository of packages to use. This is how Debian handles cargo dependencies, and I think Nix does something similar as well.
  - cycloptic 7y ago
    This only works if you keep your list of dependencies small and limited to the ones that are stable and have been tested and packaged. It still seems common to get in npm-like situations with Rust programs where typing "cargo build" installs hundreds of packages. This is what I mean by the ecosystem lacking maturity.

JoshTriplett 7y ago

Distribution maintainers have already figured out how to package Rust in a modular fashion and build without touching the network. Cargo specifically provides facilities to help distribution maintainers provide dependencies themselves. Based on those facilities, I built the initial prototype of Rust packaging for Debian ("debcargo"), and the current Debian Rust maintainers have made further huge advances in packaging. So no, Cargo doesn't "sidestep" distribution packaging, it works with distribution packaging. Also, neither the standard library nor the compiler have "churn"; we're very careful to make sure old code still compiles with current Rust.

daurnimator 7y ago
On arch we haven't yet. We use Cargo.lock files to get reproducible builds, but trying to get rust to follow normal practices has been difficult.
cycloptic 7y ago
Yes, debcargo is a great solution. Thank you for writing that. And really, I mean it, it's awesome. But my point (which you seem to be agreeing with) is that we have a while to wait while the distro people do their thing. Not just Debian. We can't just take random packages off github and put them in any distro, there is more to it than just running debcargo. >Also, neither the standard library nor the compiler have "churn"; we're very careful to make sure old code still compiles with current Rust. When did this happen? Last I checked Rust still had no LTS policy and there were no stability guarantees around the syntax, ABI, API, compiler flags, Cargo file format, etc. If some policy has changed here, please link me to any info.
- JoshTriplett 7y ago
  > Yes, debcargo is a great solution. Thank you for writing that. And really, I mean it, it's awesome. Just to reiterate: I wrote the initial prototype, and massive credit goes to the current maintainers who picked it up and ran with it into production. > there were no stability guarantees around the syntax, ABI, API, compiler flags, Cargo file format, etc. There are absolutely stability guarantees around syntax, around compiler flags, around API, and around the Cargo file format. Code from old Rust and old Cargo will build and run with new Rust and new Cargo. We take a great deal of care to preserve that stability guarantee. The policies here have remained essentially the same since 1.0 (https://blog.rust-lang.org/2014/10/30/Stability.html https://blog.rust-lang.org/2014/10/30/Stability.html), though we've since taken further care to specify them more precisely and to stabilize further areas. (Source: I'm one of the leads of the Rust language team, and I'm on the Rust cargo team. Other Rust teams, such as the Rust library team, follow the same principles.)

unlinked_dll 7y ago

(as of iirc 1.39) Cargo supports custom registries on stable. That means you can pull your dep to whatever hosting solution you want and leave it. As well, pulling from a random server isn't the problem. Trusting a version to be a particular commit is, which crates.io handles as well as Cargo via cargo publish and the lock file. If you pull a dep, the lock file will guarantee that whatever you pull is going to be identical to what was published (assuming you didn't add a git field to the dependencies list or say version = "*"). As for churn in the std or dynamic linking, I've never had an issue using Rust in production and getting reproducible builds otherwise. Same is true for Clang or GCC/stdlib. APIs aren't being changed, binaries may shift, but unless you're on nightly you're probably not going to see anything break - just get better. But you can handle that, just don't update the compiler until you're ready.