4 ms·

It is also about how easy rust makes it to use libraries, and depend upon external libraries as a part of your public API. C in particular makes this pretty in

It is also about how easy rust makes it to use libraries, and depend upon external libraries as a part of your public API. C in particular makes this pretty inconvenient, so a lot of projects include data structures and things that otherwise might be pulled in via library themselves. That lends itself towards less reliance on external deps but also more internal stability. At the very least I can say, that experiences I've had that are common in rust, where I have dependency X needing to be upgraded in lockstep (the same version) across multiple external dependencies with different maintainers. Are things that I've basically never experienced with C based languages. At least I don't think it is just that things need time to stabilize, but also just the packaging system causes network effects which you don't get the joy of experiencing if you don't have one.

cmrdporcupine 2y ago
This is the right answer. Crates.io comes with a whole mixed bag of experiences. Increases initial project development velocity at the expense of long-term velocity (and sanity). After 2.5 years of doing Rust full-time it's the one thing I'd look back and say I'd rather see done entirely different.
- throwaway17_17 2y ago
  What do you think some of the alternatives for library distribution and management are that would be more successful. I am an unabashed proponent of the C style manage t dependencies yourself in a manner consistent for your own work, but many people are 100% adverse to that position. Do you think a middle ground would be possible?
  - Ygg2 2y ago
    Then vendor your dependencies in Rust. If you don't want libraries to change and the depenency maintainer won't commit to stable API, you have to vendor it.
    - marvin-hansen 2y ago
      I am currently working on a Rust project of about 70 crates that builds with Bazel. I ended up with vendoring everything because of very subtle bugs in rules_rust for Bazel that caught me off guard and seem to happen on non vendored deps. Fun fact, all Rust code in rules_rust is vendored by default so it's clearly the way to go. Also, previously I had weird quirks occuring randomly, but after having switched to vendored deps, everything works stable, reliable, and as expected. I don't agree with the fast pace of how often certain crates produce breaking changes, but I found that with Bazel at least stuff works all the time regardless. Also, builds are noticably faster because downloading and building sub dependencies can take considerably time in large projects.
  - cmrdporcupine 2y ago
    I prefer "vendored" self managed dependencies as well. However the Rust ecosystem will fight fight fight you in this regard. I am leaning towards starting future Rust personal projects on buck or bazel or gn, instead of Cargo. And checked-in vendored dependencies. It's not a popular position, but I have seen the crates explosion in larger projects go so wrong... And 10 years at Google taught me that vendored third party deps is an entirely reasonable approach for that.
    - steveklabnik 2y ago
      Cargo includes a vendor command. You can do the checked-in vendored dependencies if you'd like. That said I am intrigued by using buck more as well, though I haven't been actually doing it yet.
      - cmrdporcupine 2y ago
        The vendoring support in Cargo isn't really what I'm looking for. It's more "download all my deps from crates.io and make local copies" and it has various limitations I kept running into which I don't have time this morning to go back and document. It really felt like an afterthought and not at all like the third_party workflow that people who have worked with the Google monorepo would be used to.
        steveklabnik 2y ago
        I am interested in what vendoring means other than “put all of my dependencies’ code in my repo” but if you don’t have time, that’s chill. I’ve only seen open source projects using buck/bazel and haven’t ever seen google3, so I’m sure I’m missing something.
        CraigJPerry 2y ago
        Not OP but a difference is in who manages the updating of a vendored dep. You don’t want to update every time there’s a release, but you don’t want to never update either. If upstream (which you’ve vendored) releases a security fix, how does your system capture and act on that event? If vendoring just means we periodically pull a copy of upstream into our repo then you can have unbounded time when you don’t know there’s a vuln and therefore haven’t considered whether you need to act. This is obv different from the situation where we all just cargo install (without —locked) and different from c style dep mgmt where we get sec fixes for many libs via our OS patching.
atoav 2y ago
Well. Rust has lock files, so breaks will only happen when you decide they are allowed to happen, by running cargo update. In C they also only happen when you update the dependencies you have. Granted, Rust makes it easier to pull on dependecies, but it also makes it easier to keep them up to date.
- Starlevel004 2y ago
  > Rust has lock files, so breaks will only happen when you decide they are allowed to happen, by running cargo update. great in theory but ``cargo install`` will straight up ignore lockfiles by default[0], meaning trying to install a binary package is a roulette of "did this tiny package update before everyone in the dependency tree noticed" [0]: https://github.com/rust-lang/cargo/issues/7169 https://github.com/rust-lang/cargo/issues/7169

cmrdporcupine 2y ago

This is the right answer. Crates.io comes with a whole mixed bag of experiences. Increases initial project development velocity at the expense of long-term velocity (and sanity). After 2.5 years of doing Rust full-time it's the one thing I'd look back and say I'd rather see done entirely different.

throwaway17_17 2y ago
What do you think some of the alternatives for library distribution and management are that would be more successful. I am an unabashed proponent of the C style manage t dependencies yourself in a manner consistent for your own work, but many people are 100% adverse to that position. Do you think a middle ground would be possible?
- Ygg2 2y ago
  Then vendor your dependencies in Rust. If you don't want libraries to change and the depenency maintainer won't commit to stable API, you have to vendor it.
  - marvin-hansen 2y ago
    I am currently working on a Rust project of about 70 crates that builds with Bazel. I ended up with vendoring everything because of very subtle bugs in rules_rust for Bazel that caught me off guard and seem to happen on non vendored deps. Fun fact, all Rust code in rules_rust is vendored by default so it's clearly the way to go. Also, previously I had weird quirks occuring randomly, but after having switched to vendored deps, everything works stable, reliable, and as expected. I don't agree with the fast pace of how often certain crates produce breaking changes, but I found that with Bazel at least stuff works all the time regardless. Also, builds are noticably faster because downloading and building sub dependencies can take considerably time in large projects.
- cmrdporcupine 2y ago
  I prefer "vendored" self managed dependencies as well. However the Rust ecosystem will fight fight fight you in this regard. I am leaning towards starting future Rust personal projects on buck or bazel or gn, instead of Cargo. And checked-in vendored dependencies. It's not a popular position, but I have seen the crates explosion in larger projects go so wrong... And 10 years at Google taught me that vendored third party deps is an entirely reasonable approach for that.
  - steveklabnik 2y ago
    Cargo includes a vendor command. You can do the checked-in vendored dependencies if you'd like. That said I am intrigued by using buck more as well, though I haven't been actually doing it yet.
    - cmrdporcupine 2y ago
      The vendoring support in Cargo isn't really what I'm looking for. It's more "download all my deps from crates.io and make local copies" and it has various limitations I kept running into which I don't have time this morning to go back and document. It really felt like an afterthought and not at all like the third_party workflow that people who have worked with the Google monorepo would be used to.
      - steveklabnik 2y ago
        I am interested in what vendoring means other than “put all of my dependencies’ code in my repo” but if you don’t have time, that’s chill. I’ve only seen open source projects using buck/bazel and haven’t ever seen google3, so I’m sure I’m missing something.
        CraigJPerry 2y ago
        Not OP but a difference is in who manages the updating of a vendored dep. You don’t want to update every time there’s a release, but you don’t want to never update either. If upstream (which you’ve vendored) releases a security fix, how does your system capture and act on that event? If vendoring just means we periodically pull a copy of upstream into our repo then you can have unbounded time when you don’t know there’s a vuln and therefore haven’t considered whether you need to act. This is obv different from the situation where we all just cargo install (without —locked) and different from c style dep mgmt where we get sec fixes for many libs via our OS patching.

atoav 2y ago

Well. Rust has lock files, so breaks will only happen when you decide they are allowed to happen, by running cargo update. In C they also only happen when you update the dependencies you have. Granted, Rust makes it easier to pull on dependecies, but it also makes it easier to keep them up to date.

Starlevel004 2y ago
> Rust has lock files, so breaks will only happen when you decide they are allowed to happen, by running cargo update. great in theory but ``cargo install`` will straight up ignore lockfiles by default[0], meaning trying to install a binary package is a roulette of "did this tiny package update before everyone in the dependency tree noticed" [0]: https://github.com/rust-lang/cargo/issues/7169 https://github.com/rust-lang/cargo/issues/7169