6 ms·

> The dependency trees for any meaty Rust project quickly become pretty horrifying. s/Rust// This is really no different from any other language. At least Ru

> The dependency trees for any meaty Rust project quickly become pretty horrifying. s/Rust// This is really no different from any other language. At least Rust, with Cargo, makes it easy to scan your dependencies. And many notable Rust projects attempt to keep third party dependencies to a minimum. C++ gives you absolutely nothing to work with. Other languages with package managers don't keep dependency trees shallow. You're holding Rust up to a standard that nothing meets.

runevault 2y ago
I'm not sure languages without a package manager being the default are nearly as bad. When it is trivial (like running a single command line or a single line in a file like cargo.toml), having a proliferation of dependencies is so easy it almost becomes guaranteed. But in languages like c++ where not everyone uses them (even if several PMs exist) adding a dependency to a library is a much bigger deal as you either have to use something like git submodules or manage them yourself OR make any user of your library go get the correct versions themselves.
- evilduck 2y ago
  > I'm not sure languages without a package manager being the default are nearly as bad. Javascript doesn't have a specified package manager.
- throwup238 2y ago
  > I'm not sure languages without a package manager being the default are nearly as bad. The autoconf or cmake list of included libraries for many non-trivial C++ projects is usually just as long as stuff in Cargo, especially when breaking up Boost, Qt, or other megaframeworks into individual libraries. They do "make any user of your library go get the correct versions themselves" but that has been far less of a problem this century thanks to OS package management.
- Lvl999Noob 2y ago
  The thing is, when adding dependencies _isn't_ trivial, you end up vendoring. You aren't gonna thoroughly test your command line parser if it is a very small feature of your overall project. A dedicated command line parser lib will (more likely than you in this case) thoroughly test their implementation.
- wiseowise 2y ago
  [flagged]
- wtetzner 2y ago
  I think it's important to figure out what you're comparing though. A language that makes it easy to pull in dependencies also encourages breaking code into separate modules. Languages that make dependency management hard tend to have larger, heavier dependencies. Two dependencies in such a language are more likely to have duplicate functionality between them, instead of sharing the functionality through another dependency. Is it better to vet many smaller dependencies, or fewer large ones that likely duplicate a lot of stuff? It depends on what those dependencies are. I don't think just looking at dependency counts is that useful. Many libraries that would be a single dependency in other languages are split into several because they are useful on their own.
troad 2y ago
> And many notable Rust projects attempt to keep third party dependencies to a minimum. I don't think this is true. The only two major Rust crates that manage to keep their dependencies light are tokio and serde, and these are highly atypical projects. For a more typical example, look at something like axum (running `cargo tree` for project with a single dependency on axum returns 121 lines). > This is really no different from any other language. > You're holding Rust up to a standard that nothing meets. Respectfully, I think you're creating a bit of a false dichotomy here. I'm not demanding perfection, I'm merely noting that I've found Rust dependency trees to grow noticeably faster than dependency trees in equivalent languages. You add two dependencies in Rust, and suddenly you have a dozen dependencies of dependencies of dependencies, including at least three different logging crates. In the world of C, which is what Rust is trying to displace, that's just not going to pass muster. Rust is a very fine language with a bit of a dependency addiction (a dependency dependency?). I honestly don't see what service it does to the language to pretend otherwise.
- alexchamberlain 2y ago
  I think the GP's point was that all modern languages (and some of the older ones) have this problem - in fact, JS is infamous for it. Therefore, I don't think it's really a pro or a con on its own - only in the context of your business and problem.
- burntsushi 2y ago
  tokio and serde are certainly not the only ones. You can put almost all of my crates into that category too. The problem with your framing is that you look at this as a "dependency addiction." But that doesn't fully explain everything. The `regex` crate is a good case study. If it were a C library, it would almost certainly have zero dependencies. But it isn't a C library. It exists in a context where I can encapsulate separately versioned libraries as dependencies with almost no impact on users of `regex`. Namely, it has two required dependencies: regex-syntax and regex-automata. It also has two optional dependencies: memchr and aho-corasick. This isn't a case of the regex crate farming out its core functionality to other projects. Indeed, it started as a single crate. And I split its code out into separately versioned crates that others can now use. And this has been a major ecosystem win: * memchr is used in all sorts of projects, and it promises to give you exactly the same implementation of substring search that the regex crate (and also ripgrep) use in your own projects. Indeed, that crate is used here! What would you do instead? If you were in C-land, you'd re-roll all of the specialized SIMD that's in memchr? For x86-64, aarch64 and wasm32 right? If you haven't done that sort of thing before, good luck. That'll be a long ramp-up time. * aho-corasick is packaged as a stand-alone Python library that is quite a bit faster than pyahocorasick: https://pypi.org/project/ahocorasick-rs/ https://pypi.org/project/ahocorasick-rs/ There's tons of other projects on crates.io relying on aho-corasick specifically, separately from how its used inside of `regex`. * regex-syntax gives you a production grade regex parser. More than that, it gives you exactly the same parser used by the regex crate. People have used this for all sorts of things, including building their own regex engine without needing to re-create the parser (which is a significant simplification). * regex-automata gives you access to all of the internal APIs of the regex engine. This is all the stuff that is too complex to put into a general purpose regex library targeting the 99% use case. As far as I know, literally no other general purpose regex engine has ever attempted this because most regex engines are written in C or C++ where you'd be laughed out of the room for suggesting it because dependency management is such a clusterfuck. Yet, this has been a big benefit to other folks. The Yara project uses it for example, and the Helix editor uses it to search discontiguous strings: https://github.com/helix-editor/helix/pull/9422 https://github.com/helix-editor/helix/pull/9422 (Instead of rolling your own regex engine, which is what I believe vim does.) This isn't dependency addiction. This is making use of separately versioned libraries to allow other projects to depend on battle tested components independent of their primary use case. Yet, if people repeat this kind of process---exposing internals like I did with the regex crate---then you wind up with a bigger dependency tree. Good dependency management is a trade-off. One the one hand, it enables the above to happen, which I think is an objectively Good Thing. But it also enables folks to depend on huge piles of code so easily that it actively discourages someone from writing their own base64 implementation. But as should be obvious, it doesn't prevent them from doing so: https://github.com/BurntSushi/ripgrep/blob/ea99421ec896fcc9a9cd5367d42a80f89316822f/crates/printer/src/jsont.rs#L227 https://github.com/BurntSushi/ripgrep/blob/ea99421ec896fcc9a... Good dependency management is Pandora's box. It has been opened and it is never going to get closed again. Just looking on and calling it an addiction isn't going to take us anywhere. Instead, let's look at it as a trade-off.
wiseowise 2y ago
And yet with Python and JS I don’t need to pull a freaking third-party library for something as basic as JSON.
- ModernMech 2y ago
  If I don't need it, it's not basic it's just useless.
- kelnos 2y ago
  Most of my Rust projects don't need a JSON parser. You probably haven't been a programmer that long if you think JSON is "basic". If it was 20 years ago, you'd be arguing for an XML parser in Rust's stdlib. In 20 years I'm sure it will be something else. File/data formats come and go. The stdlib of a systems programming language shouldn't be taking on a forever maintenance burden for something that likely won't be in widespread use for all that long.
- wtetzner 2y ago
  Nope, instead it's pulled into every project whether you need it or not.
  - wiseowise 2y ago
    Pulled how?
devjab 2y ago
It’s different from Go, but then, Go is probably not the language you’re going to replace Rust with. (I know it’s exactly different from Go’s dependency management, but you frankly rarely need any thing outside of the STL in Go.)
rafaelmn 2y ago
> This is really no different from any other language. There are languages with big standard libraries and first party frameworks. I can build a complex web app in C# using only packages published by Microsoft in ASP.NET and EF. Python ships with a lot of "batteries included". Not saying I expect that from rust considering the funding/team size discrepancy and language targets - but I disagree that every language is same in this regard - JS/Node is notoriously bad, Rust is around C++ level, and plenty of higher level languages have first pary/standard library stacks.
- Ygg2 2y ago
  Moment you add something to std it's effectively died. Strong backwards guarantees will choke development of library. Python std is where libraries go to die.
  - rafaelmn 2y ago
    Who says that's a bad thing ? Sometimes things are good enough and just need to be vetted and maintained.
    - Ygg2 2y ago
      Rarely that's ever true. Hardware and software assumption will change over time. E.g. Simd is rare and not available to 99% of hardware has it, pointer is 16/32/64 bit wide, that API is available, SGML/JSON/XML is all parsing you will ever need.

runevault 2y ago

I'm not sure languages without a package manager being the default are nearly as bad. When it is trivial (like running a single command line or a single line in a file like cargo.toml), having a proliferation of dependencies is so easy it almost becomes guaranteed. But in languages like c++ where not everyone uses them (even if several PMs exist) adding a dependency to a library is a much bigger deal as you either have to use something like git submodules or manage them yourself OR make any user of your library go get the correct versions themselves.

evilduck 2y ago
> I'm not sure languages without a package manager being the default are nearly as bad. Javascript doesn't have a specified package manager.
throwup238 2y ago
> I'm not sure languages without a package manager being the default are nearly as bad. The autoconf or cmake list of included libraries for many non-trivial C++ projects is usually just as long as stuff in Cargo, especially when breaking up Boost, Qt, or other megaframeworks into individual libraries. They do "make any user of your library go get the correct versions themselves" but that has been far less of a problem this century thanks to OS package management.
Lvl999Noob 2y ago
The thing is, when adding dependencies _isn't_ trivial, you end up vendoring. You aren't gonna thoroughly test your command line parser if it is a very small feature of your overall project. A dedicated command line parser lib will (more likely than you in this case) thoroughly test their implementation.
wiseowise 2y ago
[flagged]
wtetzner 2y ago
I think it's important to figure out what you're comparing though. A language that makes it easy to pull in dependencies also encourages breaking code into separate modules. Languages that make dependency management hard tend to have larger, heavier dependencies. Two dependencies in such a language are more likely to have duplicate functionality between them, instead of sharing the functionality through another dependency. Is it better to vet many smaller dependencies, or fewer large ones that likely duplicate a lot of stuff? It depends on what those dependencies are. I don't think just looking at dependency counts is that useful. Many libraries that would be a single dependency in other languages are split into several because they are useful on their own.

troad 2y ago

> And many notable Rust projects attempt to keep third party dependencies to a minimum. I don't think this is true. The only two major Rust crates that manage to keep their dependencies light are tokio and serde, and these are highly atypical projects. For a more typical example, look at something like axum (running `cargo tree` for project with a single dependency on axum returns 121 lines). > This is really no different from any other language. > You're holding Rust up to a standard that nothing meets. Respectfully, I think you're creating a bit of a false dichotomy here. I'm not demanding perfection, I'm merely noting that I've found Rust dependency trees to grow noticeably faster than dependency trees in equivalent languages. You add two dependencies in Rust, and suddenly you have a dozen dependencies of dependencies of dependencies, including at least three different logging crates. In the world of C, which is what Rust is trying to displace, that's just not going to pass muster. Rust is a very fine language with a bit of a dependency addiction (a dependency dependency?). I honestly don't see what service it does to the language to pretend otherwise.

alexchamberlain 2y ago
I think the GP's point was that all modern languages (and some of the older ones) have this problem - in fact, JS is infamous for it. Therefore, I don't think it's really a pro or a con on its own - only in the context of your business and problem.
burntsushi 2y ago
tokio and serde are certainly not the only ones. You can put almost all of my crates into that category too. The problem with your framing is that you look at this as a "dependency addiction." But that doesn't fully explain everything. The `regex` crate is a good case study. If it were a C library, it would almost certainly have zero dependencies. But it isn't a C library. It exists in a context where I can encapsulate separately versioned libraries as dependencies with almost no impact on users of `regex`. Namely, it has two required dependencies: regex-syntax and regex-automata. It also has two optional dependencies: memchr and aho-corasick. This isn't a case of the regex crate farming out its core functionality to other projects. Indeed, it started as a single crate. And I split its code out into separately versioned crates that others can now use. And this has been a major ecosystem win: * memchr is used in all sorts of projects, and it promises to give you exactly the same implementation of substring search that the regex crate (and also ripgrep) use in your own projects. Indeed, that crate is used here! What would you do instead? If you were in C-land, you'd re-roll all of the specialized SIMD that's in memchr? For x86-64, aarch64 and wasm32 right? If you haven't done that sort of thing before, good luck. That'll be a long ramp-up time. * aho-corasick is packaged as a stand-alone Python library that is quite a bit faster than pyahocorasick: https://pypi.org/project/ahocorasick-rs/ https://pypi.org/project/ahocorasick-rs/ There's tons of other projects on crates.io relying on aho-corasick specifically, separately from how its used inside of `regex`. * regex-syntax gives you a production grade regex parser. More than that, it gives you exactly the same parser used by the regex crate. People have used this for all sorts of things, including building their own regex engine without needing to re-create the parser (which is a significant simplification). * regex-automata gives you access to all of the internal APIs of the regex engine. This is all the stuff that is too complex to put into a general purpose regex library targeting the 99% use case. As far as I know, literally no other general purpose regex engine has ever attempted this because most regex engines are written in C or C++ where you'd be laughed out of the room for suggesting it because dependency management is such a clusterfuck. Yet, this has been a big benefit to other folks. The Yara project uses it for example, and the Helix editor uses it to search discontiguous strings: https://github.com/helix-editor/helix/pull/9422 https://github.com/helix-editor/helix/pull/9422 (Instead of rolling your own regex engine, which is what I believe vim does.) This isn't dependency addiction. This is making use of separately versioned libraries to allow other projects to depend on battle tested components independent of their primary use case. Yet, if people repeat this kind of process---exposing internals like I did with the regex crate---then you wind up with a bigger dependency tree. Good dependency management is a trade-off. One the one hand, it enables the above to happen, which I think is an objectively Good Thing. But it also enables folks to depend on huge piles of code so easily that it actively discourages someone from writing their own base64 implementation. But as should be obvious, it doesn't prevent them from doing so: https://github.com/BurntSushi/ripgrep/blob/ea99421ec896fcc9a9cd5367d42a80f89316822f/crates/printer/src/jsont.rs#L227 https://github.com/BurntSushi/ripgrep/blob/ea99421ec896fcc9a... Good dependency management is Pandora's box. It has been opened and it is never going to get closed again. Just looking on and calling it an addiction isn't going to take us anywhere. Instead, let's look at it as a trade-off.

wiseowise 2y ago

And yet with Python and JS I don’t need to pull a freaking third-party library for something as basic as JSON.

ModernMech 2y ago
If I don't need it, it's not basic it's just useless.
kelnos 2y ago
Most of my Rust projects don't need a JSON parser. You probably haven't been a programmer that long if you think JSON is "basic". If it was 20 years ago, you'd be arguing for an XML parser in Rust's stdlib. In 20 years I'm sure it will be something else. File/data formats come and go. The stdlib of a systems programming language shouldn't be taking on a forever maintenance burden for something that likely won't be in widespread use for all that long.
wtetzner 2y ago
Nope, instead it's pulled into every project whether you need it or not.
- wiseowise 2y ago
  Pulled how?

devjab 2y ago

It’s different from Go, but then, Go is probably not the language you’re going to replace Rust with. (I know it’s exactly different from Go’s dependency management, but you frankly rarely need any thing outside of the STL in Go.)

rafaelmn 2y ago

> This is really no different from any other language. There are languages with big standard libraries and first party frameworks. I can build a complex web app in C# using only packages published by Microsoft in ASP.NET and EF. Python ships with a lot of "batteries included". Not saying I expect that from rust considering the funding/team size discrepancy and language targets - but I disagree that every language is same in this regard - JS/Node is notoriously bad, Rust is around C++ level, and plenty of higher level languages have first pary/standard library stacks.

Ygg2 2y ago
Moment you add something to std it's effectively died. Strong backwards guarantees will choke development of library. Python std is where libraries go to die.
- rafaelmn 2y ago
  Who says that's a bad thing ? Sometimes things are good enough and just need to be vetted and maintained.
  - Ygg2 2y ago
    Rarely that's ever true. Hardware and software assumption will change over time. E.g. Simd is rare and not available to 99% of hardware has it, pointer is 16/32/64 bit wide, that API is available, SGML/JSON/XML is all parsing you will ever need.