8 ms·

I feel like the author really overstates how hard it is to make a new crate. Making a cargo.toml is simple and frankly you don’t do it so often as to be a burde

I feel like the author really overstates how hard it is to make a new crate. Making a cargo.toml is simple and frankly you don’t do it so often as to be a burden. Encouraging cyclic dependencies on the other hand “because it is easy” seems like a lazy cop out.

virtualritz 3y ago
> I feel like the author really overstates how hard it is to make a new crate. I read it more as the maintenance overhead when publishing. You need to bump version, check dependency versions etc. I'm speaking from personal experience here. The more I split my crates into sub-crates, the longer the release/publish process takes. There are some parts you can automate with e.g. cargo-release but I now think a lot harder before cutting a release to crates.io. I also have the gut feeling this is happening throughout the ecosystem. I have a lot more overlays in my Cargo.tomls now than I used to, two years ago. I.e. there are PRs and fixes in the repo. that you need but the author hasn't cut a release. This may be coincidence ofc but I can't help but think there may be a connection.
jchw 3y ago
Well, they seem to be cognizant of that, but they have a point: compared to simply creating a folder, it is QUITE a lot more effort. It's like the difference between zero and one. That having been said, I think my bigger issue with this is that this actually gets to be annoying after you do it, not just while doing it. Go manages external dependencies on the module level, which is a level higher than packages. For Rust, it's crates. So a repo with many crates becomes pretty cumbersome. (And a bit moreso if you're trying to maintain a Nix package for said repo, but that's partly the fault of Nix. Only partly, because I actually think Rust's toolchain makes the job of Nix rather difficult...)
- mikepurvis 3y ago
  As a moderate Nix user/packager, I feel like the ideal model for ecosystem packaging in Nix is poetry2nix, where the existing lockfile hashes are able to be used directly to create pure Nix builds, with no need to generate code, pre-download anything, or have opaque "deps" blobs (like with Bazel). As far as I can tell, all or most of these requirements are met by Cargo lockfiles as well, so what is the gap that makes Cargo/Rust difficult to do well in Nix?
  - jchw 3y ago
    The main issue I've come across is that when dealing with multi-crate projects, it is not uncommon to have two versions of the exact same version number of a Rust crate in two different dependencies. Due to a limitation in the machinery, this is painful. The other thing that's weird is that you have to embed the lockfile into the Nix derivative, at least in some cases. I actually can't remember why that has to be done, but presumably there is a good reason. I think most multi-crate projects are only sorta accidentally including multiple versions of the same crate, which is why it's rather unfortunate that there isn't some abstraction between crate and module.
flurker 3y ago
Making Cargo.toml is annoying enough especially if we're factoring things out from an existing crate to a separate one: obviously we'll need to specify all the deps that the code is already using, and also possibly remove some deps from the old crate's Cargo.toml, and it can take a good chunk of time just to put it together. I mean it's nothing that we can't deal with, but it does slow things down for no good reason.
- inferiorhuman 3y ago
  Copy over all the dependencies and then run something like this: https://blog.benj.me/2022/04/27/cargo-machete/ https://blog.benj.me/2022/04/27/cargo-machete/
  - cratermoon 3y ago
    Maybe something like that should be built into the Rust toolchain, rather than being a third-party tool?
    - db48x 3y ago
      Maybe one day. In the mean time, it is best for many people to invent many tools and then find out which work the best for the most people.
      - cratermoon 3y ago
        Agreed. Is there a community process for proposing changes and enhancements to the tooling?
        steveklabnik 3y ago
        There is, Cargo uses the RFC process for big changes just like rustc. However, the Cargo team has been stretched a bit thin; back in April they added two new team members, and so things look like they're going in the right direction again: https://blog.rust-lang.org/inside-rust/2023/04/06/cargo-new-members.html https://blog.rust-lang.org/inside-rust/2023/04/06/cargo-new-...
kjuulh 3y ago
It is super easy to do. Cargo init --lib crates/something, done.. xD it is pretty much identical to go. I've got other gripes with cargo. - A go mod download equivalent would be nice so that we can actually reliably cache dependencies in ci. - More sane linking tooling when statically compiling, eg. Openssl. - Build.rs magic at compile time. - No good way to publish a workspace of crates without actually having to publish everything. - General workspace versioning unfriendliness. It is tricky to bump versions correctly in a workspace. Probably some more. Most of these already have upstream issues but most have been stale for a while
- Ygg2 3y ago
  To be fair, it seems Cargo team has been understaffed for a long time. So developement has been kinda stalling. They are trying to fix it.
  - kjuulh 3y ago
    Yeah, it maybe wasn't the greatest to throw blame at them, I really enjoy using rust and cargo. So I just want it to be the best it can be. Maybe I should look into what I can do to help them solve some of these issues =D
- inferiorhuman 3y ago
  A go mod download equivalent would be nice so that we can actually reliably cache dependencies in ci. What's wrong with caching the target and ~/.cargo directories?
  - kjuulh 3y ago
    you have to include your source to be able to build your dependencies, this is not great for Docker builds, or just hashing in general. Cargo-chef tries to solve this, by only Including Cargo.toml and .lock, and creating main.rs and lib.rs files when it runs, it is a bit clunky though. Basically you don't want a source change to invalidate your dependency fetching, which can occur if you use cargo fetch and just caching the target, especially with docker, as it doesn't have granular enough caching mechanisms. Caching ~/.cargo is great tho =D
ilyt 3y ago
I do dislike how it forces entirety of code to be under single file. I can't for example shove all of the definitions into single file then use it in other files without using sub-modules for no good reason, I much prefer "module = directory" approach to "module = file"
- inferiorhuman 3y ago
  I much prefer "module = directory" approach to "module = file" https://doc.rust-lang.org/reference/items/modules.html https://doc.rust-lang.org/reference/items/modules.html
  - ilyt 3y ago
    Why are you linking a document showing module need to be single file ?
epage 3y ago
There are some efforts for lowering the overhead and would be interesting to explore how we could further do it. First, we allow sharing `Cargo.toml` fields across the workspace since 1.64.0 [0]. In the upcoming 1.71.0, we extend this so that `cargo new` will automatically inherit fields when creating new `Cargo.toml` files [1]. From a different angle, we are also adding support for single-file packages though this is more meant for binaries [2] [0] https://doc.rust-lang.org/cargo/reference/workspaces.html#the-package-table https://doc.rust-lang.org/cargo/reference/workspaces.html#th... [1] https://github.com/rust-lang/cargo/pull/12069 https://github.com/rust-lang/cargo/pull/12069 [2] https://github.com/rust-lang/rfcs/pull/3424 https://github.com/rust-lang/rfcs/pull/3424
renewiltord 3y ago
It's quite annoying in an ergonomic sense. Dependency management becomes harder, etc. For instance, if you make more crates and they all use the same dependency, you have to manage it all in the multiple Cargo.toml files. But overall, they're just two different systems with two different tradeoffs. I prefer writing Rust and I prefer the way I specify deps in Rust, but I prefer the ergonomics of packages in Go.
cratermoon 3y ago
> Making a cargo.toml is simple and frankly you don’t do it so often as to be a burden. But is it so simple that it can be done often without having to go back to the docs and look up something, then making sure everything in the crate makes the build system happy? We get better at doing things the more often we do them, of course, so there's some inflection point where "often enough to remember how to do it" and "simple enough to be able to recall" cross. With Go, making a new module is just creating a new directory, something programmers have done countless times, and requires no cooperation from the build tool. The only requirement is that files in directory have the same package declaration.
zemo 3y ago
right but if you have two crates you have to list the common dependencies twice, and you have to either keep them in sync manually or put the crates into a workspace and then use workspace dependencies, so now you have to declare the dependent crate three times.

virtualritz 3y ago

> I feel like the author really overstates how hard it is to make a new crate. I read it more as the maintenance overhead when publishing. You need to bump version, check dependency versions etc. I'm speaking from personal experience here. The more I split my crates into sub-crates, the longer the release/publish process takes. There are some parts you can automate with e.g. cargo-release but I now think a lot harder before cutting a release to crates.io. I also have the gut feeling this is happening throughout the ecosystem. I have a lot more overlays in my Cargo.tomls now than I used to, two years ago. I.e. there are PRs and fixes in the repo. that you need but the author hasn't cut a release. This may be coincidence ofc but I can't help but think there may be a connection.

jchw 3y ago

Well, they seem to be cognizant of that, but they have a point: compared to simply creating a folder, it is QUITE a lot more effort. It's like the difference between zero and one. That having been said, I think my bigger issue with this is that this actually gets to be annoying after you do it, not just while doing it. Go manages external dependencies on the module level, which is a level higher than packages. For Rust, it's crates. So a repo with many crates becomes pretty cumbersome. (And a bit moreso if you're trying to maintain a Nix package for said repo, but that's partly the fault of Nix. Only partly, because I actually think Rust's toolchain makes the job of Nix rather difficult...)

mikepurvis 3y ago
As a moderate Nix user/packager, I feel like the ideal model for ecosystem packaging in Nix is poetry2nix, where the existing lockfile hashes are able to be used directly to create pure Nix builds, with no need to generate code, pre-download anything, or have opaque "deps" blobs (like with Bazel). As far as I can tell, all or most of these requirements are met by Cargo lockfiles as well, so what is the gap that makes Cargo/Rust difficult to do well in Nix?
- jchw 3y ago
  The main issue I've come across is that when dealing with multi-crate projects, it is not uncommon to have two versions of the exact same version number of a Rust crate in two different dependencies. Due to a limitation in the machinery, this is painful. The other thing that's weird is that you have to embed the lockfile into the Nix derivative, at least in some cases. I actually can't remember why that has to be done, but presumably there is a good reason. I think most multi-crate projects are only sorta accidentally including multiple versions of the same crate, which is why it's rather unfortunate that there isn't some abstraction between crate and module.

flurker 3y ago

Making Cargo.toml is annoying enough especially if we're factoring things out from an existing crate to a separate one: obviously we'll need to specify all the deps that the code is already using, and also possibly remove some deps from the old crate's Cargo.toml, and it can take a good chunk of time just to put it together. I mean it's nothing that we can't deal with, but it does slow things down for no good reason.

inferiorhuman 3y ago
Copy over all the dependencies and then run something like this: https://blog.benj.me/2022/04/27/cargo-machete/ https://blog.benj.me/2022/04/27/cargo-machete/
- cratermoon 3y ago
  Maybe something like that should be built into the Rust toolchain, rather than being a third-party tool?
  - db48x 3y ago
    Maybe one day. In the mean time, it is best for many people to invent many tools and then find out which work the best for the most people.
    - cratermoon 3y ago
      Agreed. Is there a community process for proposing changes and enhancements to the tooling?
      - steveklabnik 3y ago
        There is, Cargo uses the RFC process for big changes just like rustc. However, the Cargo team has been stretched a bit thin; back in April they added two new team members, and so things look like they're going in the right direction again: https://blog.rust-lang.org/inside-rust/2023/04/06/cargo-new-members.html https://blog.rust-lang.org/inside-rust/2023/04/06/cargo-new-...

kjuulh 3y ago

It is super easy to do. Cargo init --lib crates/something, done.. xD it is pretty much identical to go. I've got other gripes with cargo. - A go mod download equivalent would be nice so that we can actually reliably cache dependencies in ci. - More sane linking tooling when statically compiling, eg. Openssl. - Build.rs magic at compile time. - No good way to publish a workspace of crates without actually having to publish everything. - General workspace versioning unfriendliness. It is tricky to bump versions correctly in a workspace. Probably some more. Most of these already have upstream issues but most have been stale for a while

Ygg2 3y ago
To be fair, it seems Cargo team has been understaffed for a long time. So developement has been kinda stalling. They are trying to fix it.
- kjuulh 3y ago
  Yeah, it maybe wasn't the greatest to throw blame at them, I really enjoy using rust and cargo. So I just want it to be the best it can be. Maybe I should look into what I can do to help them solve some of these issues =D
inferiorhuman 3y ago
A go mod download equivalent would be nice so that we can actually reliably cache dependencies in ci. What's wrong with caching the target and ~/.cargo directories?
- kjuulh 3y ago
  you have to include your source to be able to build your dependencies, this is not great for Docker builds, or just hashing in general. Cargo-chef tries to solve this, by only Including Cargo.toml and .lock, and creating main.rs and lib.rs files when it runs, it is a bit clunky though. Basically you don't want a source change to invalidate your dependency fetching, which can occur if you use cargo fetch and just caching the target, especially with docker, as it doesn't have granular enough caching mechanisms. Caching ~/.cargo is great tho =D

ilyt 3y ago

I do dislike how it forces entirety of code to be under single file. I can't for example shove all of the definitions into single file then use it in other files without using sub-modules for no good reason, I much prefer "module = directory" approach to "module = file"

inferiorhuman 3y ago
I much prefer "module = directory" approach to "module = file" https://doc.rust-lang.org/reference/items/modules.html https://doc.rust-lang.org/reference/items/modules.html
- ilyt 3y ago
  Why are you linking a document showing module need to be single file ?

epage 3y ago

There are some efforts for lowering the overhead and would be interesting to explore how we could further do it. First, we allow sharing `Cargo.toml` fields across the workspace since 1.64.0 [0]. In the upcoming 1.71.0, we extend this so that `cargo new` will automatically inherit fields when creating new `Cargo.toml` files [1]. From a different angle, we are also adding support for single-file packages though this is more meant for binaries [2] [0] https://doc.rust-lang.org/cargo/reference/workspaces.html#the-package-table https://doc.rust-lang.org/cargo/reference/workspaces.html#th... [1] https://github.com/rust-lang/cargo/pull/12069 https://github.com/rust-lang/cargo/pull/12069 [2] https://github.com/rust-lang/rfcs/pull/3424 https://github.com/rust-lang/rfcs/pull/3424

renewiltord 3y ago

It's quite annoying in an ergonomic sense. Dependency management becomes harder, etc. For instance, if you make more crates and they all use the same dependency, you have to manage it all in the multiple Cargo.toml files. But overall, they're just two different systems with two different tradeoffs. I prefer writing Rust and I prefer the way I specify deps in Rust, but I prefer the ergonomics of packages in Go.

cratermoon 3y ago

> Making a cargo.toml is simple and frankly you don’t do it so often as to be a burden. But is it so simple that it can be done often without having to go back to the docs and look up something, then making sure everything in the crate makes the build system happy? We get better at doing things the more often we do them, of course, so there's some inflection point where "often enough to remember how to do it" and "simple enough to be able to recall" cross. With Go, making a new module is just creating a new directory, something programmers have done countless times, and requires no cooperation from the build tool. The only requirement is that files in directory have the same package declaration.

zemo 3y ago

right but if you have two crates you have to list the common dependencies twice, and you have to either keep them in sync manually or put the crates into a workspace and then use workspace dependencies, so now you have to declare the dependent crate three times.