4 ms·

I consciously remove and rewrite various dependencies at work, but I feel it's only a half of the whole story because either 1K or 4M lines of code seem to be e

I consciously remove and rewrite various dependencies at work, but I feel it's only a half of the whole story because either 1K or 4M lines of code seem to be equally inaccurate estimates for the appropriate number of LoC for this project. It seems that most dependencies of cargo-watch are pulled from three direct requirements: clap, cargo_metadata and watchexec. Clap would pull lots of CLI things that would be naturally platform-dependent, while cargo_metadata will surely pull most serde stuffs. Watchexec does have a room for improvement though, because it depends on command-group (maintained in the same org) which unconditionally requires Tokio! Who would have expected that? Once watchexec got improved on that aspect however, I think these requirements are indeed necessary for the project's goal and any further dependency removal will probably come with some downsides. A bigger problem here is that you can't easily fix other crates' excessive dependencies. Watchexec can be surely improved, but what if other crates are stuck at the older version of watchexec? There are some cases where you can just tweak Cargo.lock to get things aligned, but generally you can't do that. You have to live with excessive and/or duplicate dependencies (not a huge problem by itself, so it's default for most people) or work around with `[patch]` sections. (Cargo is actually in a better shape given that the second option is even possible at all!) In my opinion there should be some easy way to define a "stand-in" for given version of crate, so that such dependency issues can be more systematically worked around. But any such solution would be a huge research problem for any existing package manager.

cmrdporcupine 2y ago
It's frustrating because the grand-daddy of build systems with automatic transitive dependency management -- Maven -- already had tools from day one to handle this kind of thing through excluded dependencies (a blunt instrument, but sometimes necessary). In my experience, [patch] doesn't cut it or compare. That, and the maven repository is moderated. Unlike crates.io. Crates.io is a real problem. No namespaces, basically unmoderated, tons of abandoned stuff. Version hell like you're talking about. I have a hard time taking it at all seriously as a professional tool. And it's only going to get worse. If I were starting a Rust project from scratch inside a commercial company at this point, I'd use Bazel or Buck or GN/Ninja and vendored dependencies. No Cargo, no crates.io.
- lifthrasiir 2y ago
  > In my experience, [patch] doesn't cut it or compare. AFAIK what Maven does is an exclusion of dependency edges, which is technically an unsafe thing to do. Cargo [patch] is a replacement of dependency vertices without affecting any edges. (Maven surely has a plugin to do that, but it's not built-in.) They are different things to start with. Also I believe that the edge exclusion as done by Maven is (not just "technically", but) really unsafe and only supported due to the lack of better alternatives. Edges are conceptually dependent to the incoming vertex, so it should be that vertex's responsibility to override problematic edges. An arbitrary removal of edges (or vertices) is much harder to track and many other systems have related pains from that. What I'm proposing here is therefore the extension of Cargo's vertex replacement: you should be able to share such replacements so that they can be systematically dealt. If my transitive dependencies contain some crate X with two different versions 1.0 and 2.0 (say), I should be able to write an adapter from 2.0 to 1.0 or vice versa, and ideally such adapter should be available from the crate author or from the community. I don't think Maven did try any such systematic solution. > That, and the maven repository is moderated. Unlike crates.io. Only the central repository is moderated by Maven. Maven is not much better than Cargo once you have more remote repositories. > Crates.io is a real problem. No namespaces, basically unmoderated, tons of abandoned stuff. Version hell like you're talking about. Namespace is not a solution for name squatting: namespace is just yet another identifier that can be squatted. If you are worried about squatting, the only effective solution is sandboxing, everything else is just moving the goal post. The very existence of remote repositories also means that you can't always moderate all the stuffs and get rid of abandoned stuffs. You have to trust repositories, just like that you have to trust crates with crates.io today.
  - pas 2y ago
    namespaces are a solution to having "authenticated groups of crates", it helps structuring and more importantly restructuring crates.
    - lifthrasiir 2y ago
      That's an intention, not the outcome. You might assume that having both `@chrono/chrono` and `@chrono/tz` shows a clear connection between them, but such connection is nothing to do with namespace (the actual crate names are `chrono` and `chrono-tz`), and any authority provided by `@chrono/` prefix is offseted by the availability of similar names like `@chrno/tz` or `chrono-tz`. The only thing namespace can prevent is names starting with the exact prefix of `@chrono/`, and that's not enough.

cmrdporcupine 2y ago

It's frustrating because the grand-daddy of build systems with automatic transitive dependency management -- Maven -- already had tools from day one to handle this kind of thing through excluded dependencies (a blunt instrument, but sometimes necessary). In my experience, [patch] doesn't cut it or compare. That, and the maven repository is moderated. Unlike crates.io. Crates.io is a real problem. No namespaces, basically unmoderated, tons of abandoned stuff. Version hell like you're talking about. I have a hard time taking it at all seriously as a professional tool. And it's only going to get worse. If I were starting a Rust project from scratch inside a commercial company at this point, I'd use Bazel or Buck or GN/Ninja and vendored dependencies. No Cargo, no crates.io.

lifthrasiir 2y ago
> In my experience, [patch] doesn't cut it or compare. AFAIK what Maven does is an exclusion of dependency edges, which is technically an unsafe thing to do. Cargo [patch] is a replacement of dependency vertices without affecting any edges. (Maven surely has a plugin to do that, but it's not built-in.) They are different things to start with. Also I believe that the edge exclusion as done by Maven is (not just "technically", but) really unsafe and only supported due to the lack of better alternatives. Edges are conceptually dependent to the incoming vertex, so it should be that vertex's responsibility to override problematic edges. An arbitrary removal of edges (or vertices) is much harder to track and many other systems have related pains from that. What I'm proposing here is therefore the extension of Cargo's vertex replacement: you should be able to share such replacements so that they can be systematically dealt. If my transitive dependencies contain some crate X with two different versions 1.0 and 2.0 (say), I should be able to write an adapter from 2.0 to 1.0 or vice versa, and ideally such adapter should be available from the crate author or from the community. I don't think Maven did try any such systematic solution. > That, and the maven repository is moderated. Unlike crates.io. Only the central repository is moderated by Maven. Maven is not much better than Cargo once you have more remote repositories. > Crates.io is a real problem. No namespaces, basically unmoderated, tons of abandoned stuff. Version hell like you're talking about. Namespace is not a solution for name squatting: namespace is just yet another identifier that can be squatted. If you are worried about squatting, the only effective solution is sandboxing, everything else is just moving the goal post. The very existence of remote repositories also means that you can't always moderate all the stuffs and get rid of abandoned stuffs. You have to trust repositories, just like that you have to trust crates with crates.io today.
- pas 2y ago
  namespaces are a solution to having "authenticated groups of crates", it helps structuring and more importantly restructuring crates.
  - lifthrasiir 2y ago
    That's an intention, not the outcome. You might assume that having both `@chrono/chrono` and `@chrono/tz` shows a clear connection between them, but such connection is nothing to do with namespace (the actual crate names are `chrono` and `chrono-tz`), and any authority provided by `@chrono/` prefix is offseted by the availability of similar names like `@chrno/tz` or `chrono-tz`. The only thing namespace can prevent is names starting with the exact prefix of `@chrono/`, and that's not enough.