7 ms·

Yeah. The transitive dependency graph for small projects can hit 100 pretty quickly, which instinctively worries me. Unlike node, however, library quality is pr

Yeah. The transitive dependency graph for small projects can hit 100 pretty quickly, which instinctively worries me. Unlike node, however, library quality is pretty good in quality and how they respect SemVer, and static type checking helps as a first line check.

cmrdporcupine 3y ago
The library quality is mostly fine -- however crates.io is basically unmoderated and it will get worse and worse overtime as things bitrot and collect dust or the namespace gets abused (already happening). The biggest problem is the exploding transitive dependency graph, including multiple versions of the same basic utilities. Even small projects can suffer from this. It's too easy to reach into the grab-bag at crates.io and pull a little utility out and use it. I've personally built very small things only to find multiple versions or implementations of e.g. sha256 or rand or whatever in my own dep graph, and it gets really hard to prune them out. At work we're fairly diligent about this but still it's exploding. And when you consider the matrix of minimum-supported toolchain versions for different projects, and if you're stuck on an older Rust toolchain, it's even more pain. Rust projects starting from scratch could consider vendoring their dependencies rather than pulling from crates.io. But support for vendored dependency in Cargo is weak/bad. You run into all sorts of nightmares especially when using workspaces. It's heresy and I'll get downvoted for it, but: More and more I lean towards using bazel or gn/ninja for fresh Rust projects, and walk away from cargo & crates entirely. Cargo (+ Crates.io) was delicious to me at first, but it's a meal that tastes worse the more I have of it. EDIT: it strikes me that I didn't say why exploding dependencies bugs me so much. There are logistical reasons for sure (competing APIs, compile fails, binary bloat, etc.) but the primary reason is: exploding complexity. Every additional dependency is now a potential problem for future me/future team.
- meindnoch 3y ago
  >Cargo (+ Crates.io) was delicious to me at first, but it's a meal that tastes worse the more I have of it. This is the first step of your journey. Next step is realizing the same about the language itself.
  - cmrdporcupine 3y ago
    :-) I've been around in the industry long enough to know there is no perfect tool. Rust itself has reasonably decent foundations, and fills a niche that was missing for me for a long time. There are no mainstream-acceptable alternatives to C++ for (GC-less) systems level programming that bring in a reasonably modern (ML-ish) type system, pattern matching, proper modules, etc. However, I also think the vast majority of Rust projects out there right now are using the wrong tool. They're using Rust to write web services and web sites, and the developers are coming from places like NodeJS, TypeScript, etc. It'd be fine, but it's I believe pushing the emphasis in the Rust language community in that direction. I worked for years in the relatively sane subset of C++ that we used at Google. It's fine, I have to do C++ here and there for work, and I could even go back to it if I had to. But I mostly like Rust, and would miss it. Zig made some decent choices, but others I don't necessarily agree with, but I'd consider it for embedded or low-level dev of various types
    - ReleaseCandidat 3y ago
      > However, I also think the vast majority of Rust projects out there right now are using the wrong tool. Of course they are, same as what had happened with (to) C++. And the people that use Rust (or C++) because it's "fast" are mostly the same that write code which would be faster if it would have been written in Go, Java (Kotlin?), or C#. I guess every generation of programmers must make the same errors.
      - cmrdporcupine 3y ago
        Yeah, I've been around this block many times. I've been in the industry since the mid-90s. The problem comes down to that people have all sorts of different motivations in why and how they use technology, and organizations and projects are difficult ships to steer, and the palette of available choices is not always what we want. I prefer not to consider other people as ignorant or stupid. They just have different motivations or reasons. But back to the topic: I do think Cargo should have started from looking at Maven as an engineering foundation rather than NPM. Maven shipped from the start with solutions to many of the problems that Cargo is sick with because they were not considered.
- ReleaseCandidat 3y ago
  > More and more I lean towards using bazel or gn/ninja for fresh Rust projects, and walk away from cargo & crates entirely. I would suggest Buck2 (which is written in Rust, so you can use the existing tooling to build it) https://buck2.build/ https://buck2.build/ https://github.com/facebook/buck2 https://github.com/facebook/buck2 Rust support is also rather good ;) https://github.com/facebookincubator/reindeer https://github.com/facebookincubator/reindeer But you need "fixups" to get many cargo packages to work with Buck 2: https://github.com/facebook/buck2/tree/main/shim/third-party/rust/fixups https://github.com/facebook/buck2/tree/main/shim/third-party...
  - cmrdporcupine 3y ago
    Interesting, I'd heard of buck and classified it as a blaze/bazel clone made by Xooglers who ended up at Facebook. But didn't realize buck2 was written in Rust. One of the specific weaknesses I found with Bazel for embedded development was support for multiple toolchains was really not great. I'm sure it's improved, but I found the gn/ninja combination much better.
    - kmarc 3y ago
      It did improve. However, I'm procrastinating right now by reading HN instead of fixing our multi-compiler requirement toolchain setup for Bazel, and I am contemplating resigning from my job and IT altogether. I hate Bazel at this point.
      - cmrdporcupine 3y ago
        I'm with you. If you find a good replacement for SW eng as a career, let me know.
- xpe 3y ago
  > Rust projects starting from scratch could consider vendoring their dependencies rather than pulling from crates.io. But support for vendored dependency in Cargo is weak/bad. You run into all sorts of nightmares especially when using workspaces. Could you point us to some relevant discussions (GitHub, etc) so some of us can weigh in and possibly contribute to this pain point?
  - anonymoushn 3y ago
    One case that can cause you trouble is that if you have a workspace that contains two different programs with deps that specify incompatible versions of another dep, you can't build either :)
  - cmrdporcupine 3y ago
    No, sorry, because these are things that have happened at work, in my paid job, not my hobby, and I have higher priority things to do than help Cargo clean up its mess right now. I'd have to go dig back through chat logs and tickets. But basically: vendored (git submodule in this case) project that was workspaced, added as a dep inside a workspace... Blew up badly with general workspace complaints. And doesn't help that at work we have to accommodate working with older toolchain versions because we ship a hardware product with that requirement.
    - xpe 3y ago
      Thanks; I get it. I was thinking maybe the open source community has discussed this in some detail. I'm not actively involved in any such discussions, but I did a quick look and found that "Documentation for cargo vendor use cases" [1] is a decent starting point to jump in. In that thread a user named 'rimutaka' has a comment titled "cargo vendor-related questions asked by the community" [2] of related conversations. [1]: https://github.com/rust-lang/cargo/issues/7520 https://github.com/rust-lang/cargo/issues/7520 [2]: https://github.com/rust-lang/cargo/issues/7520#issuecomment-1979895905 https://github.com/rust-lang/cargo/issues/7520#issuecomment-...
      - deleted 3y ago
        [deleted]
- zozbot234 3y ago
  > But support for vendored dependency in Cargo is weak/bad. Could you be clearer about this? What's wrong with cargo vendor?
  - cmrdporcupine 3y ago
    I comment on this below, but just try doing a path = "" dep to a 3rd party workspaced project inside a workspaced project and see what happens. I'd have to dig through old chat logs to find the specific explosions, but cargo did not like this. (It's possible it's been fixed in recent Cargo versions, but that's not available to me.) But in the end, cargo is a tool primarily designed for building crates that depend on crates in crates.io. Even workspaces themselves are a bit of a half-baked afterthought. But cargo really should have started by looking at Maven for inspiration -- a) a moderated repository and b) Maven has much more expressive support for version restrictions etc. Instead crates.io feels like it's going the way of the world npm. Anyways, sorry if this sounds like "get off my lawn", I'm grouchy today.
    - zozbot234 3y ago
      AIUI, this is not how you use vendored dependencies in cargo. You're supposed to edit the .cargo/config.toml for your current project with the output 'cargo vendor' provides.
      - cmrdporcupine 3y ago
        I believe I did in fact do that as well and ran into the same issues. But I could revisit.
formerly_proven 3y ago
Two dozen dependencies or one and a half? └── num v0.4.1 ├── num-bigint v0.4.4 │ ├── num-integer v0.1.46 │ │ └── num-traits v0.2.18 │ │ [build-dependencies] │ │ └── autocfg v1.1.0 │ └── num-traits v0.2.18 (*) │ [build-dependencies] │ └── autocfg v1.1.0 ├── num-complex v0.4.5 │ └── num-traits v0.2.18 (*) ├── num-integer v0.1.46 (*) ├── num-iter v0.1.44 │ ├── num-integer v0.1.46 (*) │ └── num-traits v0.2.18 (*) │ [build-dependencies] │ └── autocfg v1.1.0 ├── num-rational v0.4.1 │ ├── num-bigint v0.4.4 (*) │ ├── num-integer v0.1.46 (*) │ └── num-traits v0.2.18 (*) │ [build-dependencies] │ └── autocfg v1.1.0 └── num-traits v0.2.18 (*)
- coldtea 3y ago
  Depends on when each starts depending on a different version

cmrdporcupine 3y ago

The library quality is mostly fine -- however crates.io is basically unmoderated and it will get worse and worse overtime as things bitrot and collect dust or the namespace gets abused (already happening). The biggest problem is the exploding transitive dependency graph, including multiple versions of the same basic utilities. Even small projects can suffer from this. It's too easy to reach into the grab-bag at crates.io and pull a little utility out and use it. I've personally built very small things only to find multiple versions or implementations of e.g. sha256 or rand or whatever in my own dep graph, and it gets really hard to prune them out. At work we're fairly diligent about this but still it's exploding. And when you consider the matrix of minimum-supported toolchain versions for different projects, and if you're stuck on an older Rust toolchain, it's even more pain. Rust projects starting from scratch could consider vendoring their dependencies rather than pulling from crates.io. But support for vendored dependency in Cargo is weak/bad. You run into all sorts of nightmares especially when using workspaces. It's heresy and I'll get downvoted for it, but: More and more I lean towards using bazel or gn/ninja for fresh Rust projects, and walk away from cargo & crates entirely. Cargo (+ Crates.io) was delicious to me at first, but it's a meal that tastes worse the more I have of it. EDIT: it strikes me that I didn't say why exploding dependencies bugs me so much. There are logistical reasons for sure (competing APIs, compile fails, binary bloat, etc.) but the primary reason is: exploding complexity. Every additional dependency is now a potential problem for future me/future team.

meindnoch 3y ago
>Cargo (+ Crates.io) was delicious to me at first, but it's a meal that tastes worse the more I have of it. This is the first step of your journey. Next step is realizing the same about the language itself.
- cmrdporcupine 3y ago
  :-) I've been around in the industry long enough to know there is no perfect tool. Rust itself has reasonably decent foundations, and fills a niche that was missing for me for a long time. There are no mainstream-acceptable alternatives to C++ for (GC-less) systems level programming that bring in a reasonably modern (ML-ish) type system, pattern matching, proper modules, etc. However, I also think the vast majority of Rust projects out there right now are using the wrong tool. They're using Rust to write web services and web sites, and the developers are coming from places like NodeJS, TypeScript, etc. It'd be fine, but it's I believe pushing the emphasis in the Rust language community in that direction. I worked for years in the relatively sane subset of C++ that we used at Google. It's fine, I have to do C++ here and there for work, and I could even go back to it if I had to. But I mostly like Rust, and would miss it. Zig made some decent choices, but others I don't necessarily agree with, but I'd consider it for embedded or low-level dev of various types
  - ReleaseCandidat 3y ago
    > However, I also think the vast majority of Rust projects out there right now are using the wrong tool. Of course they are, same as what had happened with (to) C++. And the people that use Rust (or C++) because it's "fast" are mostly the same that write code which would be faster if it would have been written in Go, Java (Kotlin?), or C#. I guess every generation of programmers must make the same errors.
    - cmrdporcupine 3y ago
      Yeah, I've been around this block many times. I've been in the industry since the mid-90s. The problem comes down to that people have all sorts of different motivations in why and how they use technology, and organizations and projects are difficult ships to steer, and the palette of available choices is not always what we want. I prefer not to consider other people as ignorant or stupid. They just have different motivations or reasons. But back to the topic: I do think Cargo should have started from looking at Maven as an engineering foundation rather than NPM. Maven shipped from the start with solutions to many of the problems that Cargo is sick with because they were not considered.
ReleaseCandidat 3y ago
> More and more I lean towards using bazel or gn/ninja for fresh Rust projects, and walk away from cargo & crates entirely. I would suggest Buck2 (which is written in Rust, so you can use the existing tooling to build it) https://buck2.build/ https://buck2.build/ https://github.com/facebook/buck2 https://github.com/facebook/buck2 Rust support is also rather good ;) https://github.com/facebookincubator/reindeer https://github.com/facebookincubator/reindeer But you need "fixups" to get many cargo packages to work with Buck 2: https://github.com/facebook/buck2/tree/main/shim/third-party/rust/fixups https://github.com/facebook/buck2/tree/main/shim/third-party...
- cmrdporcupine 3y ago
  Interesting, I'd heard of buck and classified it as a blaze/bazel clone made by Xooglers who ended up at Facebook. But didn't realize buck2 was written in Rust. One of the specific weaknesses I found with Bazel for embedded development was support for multiple toolchains was really not great. I'm sure it's improved, but I found the gn/ninja combination much better.
  - kmarc 3y ago
    It did improve. However, I'm procrastinating right now by reading HN instead of fixing our multi-compiler requirement toolchain setup for Bazel, and I am contemplating resigning from my job and IT altogether. I hate Bazel at this point.
    - cmrdporcupine 3y ago
      I'm with you. If you find a good replacement for SW eng as a career, let me know.
xpe 3y ago
> Rust projects starting from scratch could consider vendoring their dependencies rather than pulling from crates.io. But support for vendored dependency in Cargo is weak/bad. You run into all sorts of nightmares especially when using workspaces. Could you point us to some relevant discussions (GitHub, etc) so some of us can weigh in and possibly contribute to this pain point?
- anonymoushn 3y ago
  One case that can cause you trouble is that if you have a workspace that contains two different programs with deps that specify incompatible versions of another dep, you can't build either :)
- cmrdporcupine 3y ago
  No, sorry, because these are things that have happened at work, in my paid job, not my hobby, and I have higher priority things to do than help Cargo clean up its mess right now. I'd have to go dig back through chat logs and tickets. But basically: vendored (git submodule in this case) project that was workspaced, added as a dep inside a workspace... Blew up badly with general workspace complaints. And doesn't help that at work we have to accommodate working with older toolchain versions because we ship a hardware product with that requirement.
  - xpe 3y ago
    Thanks; I get it. I was thinking maybe the open source community has discussed this in some detail. I'm not actively involved in any such discussions, but I did a quick look and found that "Documentation for cargo vendor use cases" [1] is a decent starting point to jump in. In that thread a user named 'rimutaka' has a comment titled "cargo vendor-related questions asked by the community" [2] of related conversations. [1]: https://github.com/rust-lang/cargo/issues/7520 https://github.com/rust-lang/cargo/issues/7520 [2]: https://github.com/rust-lang/cargo/issues/7520#issuecomment-1979895905 https://github.com/rust-lang/cargo/issues/7520#issuecomment-...
    - deleted 3y ago
      [deleted]
zozbot234 3y ago
> But support for vendored dependency in Cargo is weak/bad. Could you be clearer about this? What's wrong with cargo vendor?
- cmrdporcupine 3y ago
  I comment on this below, but just try doing a path = "" dep to a 3rd party workspaced project inside a workspaced project and see what happens. I'd have to dig through old chat logs to find the specific explosions, but cargo did not like this. (It's possible it's been fixed in recent Cargo versions, but that's not available to me.) But in the end, cargo is a tool primarily designed for building crates that depend on crates in crates.io. Even workspaces themselves are a bit of a half-baked afterthought. But cargo really should have started by looking at Maven for inspiration -- a) a moderated repository and b) Maven has much more expressive support for version restrictions etc. Instead crates.io feels like it's going the way of the world npm. Anyways, sorry if this sounds like "get off my lawn", I'm grouchy today.
  - zozbot234 3y ago
    AIUI, this is not how you use vendored dependencies in cargo. You're supposed to edit the .cargo/config.toml for your current project with the output 'cargo vendor' provides.
    - cmrdporcupine 3y ago
      I believe I did in fact do that as well and ran into the same issues. But I could revisit.

formerly_proven 3y ago

Two dozen dependencies or one and a half? └── num v0.4.1 ├── num-bigint v0.4.4 │ ├── num-integer v0.1.46 │ │ └── num-traits v0.2.18 │ │ [build-dependencies] │ │ └── autocfg v1.1.0 │ └── num-traits v0.2.18 (*) │ [build-dependencies] │ └── autocfg v1.1.0 ├── num-complex v0.4.5 │ └── num-traits v0.2.18 (*) ├── num-integer v0.1.46 (*) ├── num-iter v0.1.44 │ ├── num-integer v0.1.46 (*) │ └── num-traits v0.2.18 (*) │ [build-dependencies] │ └── autocfg v1.1.0 ├── num-rational v0.4.1 │ ├── num-bigint v0.4.4 (*) │ ├── num-integer v0.1.46 (*) │ └── num-traits v0.2.18 (*) │ [build-dependencies] │ └── autocfg v1.1.0 └── num-traits v0.2.18 (*)

coldtea 3y ago
Depends on when each starts depending on a different version