3 ms·

> That absolutely dwarfs the number of any other language I've used. Rust has this flaw too. Last time I wanted to compile a simple project using actix-web and

by Zababa 5y ago

> That absolutely dwarfs the number of any other language I've used. Rust has this flaw too. Last time I wanted to compile a simple project using actix-web and probably a database library, I had 200 crates to compile. In both cases I think it's due partially to preferring small packages/crates.

zozbot234 5y ago
It's ultimately Rust's reliance on generic code that forces us to deal with so many packages at build time. In idiomatic C/C++ much of the reusable code you compile ultimately turns into reusable shared objects, that sit in binary form on your fs with their corresponding include files. This is only possible in rare cases with Rust, because the "library" equivalents (crates) don't know how their generic types and code will be instantiated downstream. Everything is the equivalent of a "header only" lib.
geodel 5y ago
Yeah, in language design they might be on cutting edge. But in software engineering they want to be close to 90's, 00's cultish software design patterns, DRY principle and so on.
deaddabe 5y ago
Note that this separation is necessary if you want to achieve somewhat parallel compilation in Rust. Every proc_macro needs its own separate crate, so I end up providing two crates for macro-based stuff. At least, the stdlib seems to have enough batteries included for the dependency count to not be as high as NodeJS projects. From my experience I would rate the count of dependencies in this order: NodeJS > Rust > Python > C/C++ Though I cannot explain why this is ordered this way between {Node, Rust, Python}. Again, batteries included? Language popularity? Beginners-friendly programming language?
- zozbot234 5y ago
  AIUI, rustc now has ad-hoc parallelization of compile within a single crate. The defined compilation unit is still the crate though, as opposed to the single file in C/C++.
- josephg 5y ago
  > I cannot explain why this is ordered this way between {Node, Rust, Python}. Again, batteries included? My vote is that its cultural. Both for "batteries included" python, and for nodejs. Its easy to forget now, but npm was extremely innovative when it first came out. It absolutely went out of its way to make adding dependencies as easy as possible. And the culture (especially in the early days of nodejs) went bananas for this philosophy of programming. I had a chat with @isaacs on a bus one time. (He was the maintainer of nodejs at the time). I asked him what he thought about package documentation, and what to do when a README isn't enough. He said that he thinks if a readme isn't enough documentation for your library, your library is probably too big and should be split up. You still see this today with packages like "isobject" (a tiny function published as a package) which still gets 53M downloads / week[1]. Correct me if I'm wrong, but as I understand it python and ruby still don't support parallel dependencies with different versions in the build tree like npm does. If a python or ruby package transitively depends on foo@1.0 and foo@2.0 then my understanding is that ruby and python lose their minds. And this problem is almost impossible for the end user to solve. So libraries like rails sort of need to be designed as one big block of software. Nodejs has no problem with this - if you do this in nodejs, npm will just quietly install both versions and node will happily wire everything up correctly. The only constraint is that no single package can have a direct dependency on both foo@1 and foo@2 at the same time. But thats not something that you ever really want to do in practice. The tooling supported this too. It was quite common to have a library with some optional features that not many people used. If you wanted to exclude them from your javascript bundle, until recently the main way to do that was by breaking your library up into small pieces so your library's consumer could pick and choose what they wanted. Building a dependency tree out of with thousands of tiny modules is exactly what nodejs is designed for. It should come as no surprise that thats what we got. [1] https://www.npmjs.com/package/isobject https://www.npmjs.com/package/isobject
  - zozbot234 5y ago
    Mentioned in a sibling comment already, but Rust has these features as well, and I think Go ended up getting them too although there was some controversy about the need for them.
- Zababa 5y ago
  I would argue that it's ordered by a mix between popularity, need for those packages and ease of installing packages. JS and Python are around the same order of magnitude of popularity, installing packages is way easier in JS than in Python (NPM might not be great but pip is hell). Rust is less popular but it's very easy to have lots of packages, plus what you mentioned about procedural macros. And the standard library, like JS, is relatively small. If you add Go to that (relatively easy to add package, less popular than Python and JS but more than Rust, lots of stuff in the standard library) which would be higher than C/C++ and lower than Python, it does seem to fit. I don't know anything about the C#/Java ecosystems, same with Ruby and Perl. I'd like to know if they fit this "model" too, that would be interesting. It could give some pointers on how to design/make a language evolve to avoid having lots of packages. I'm not sure about the standard library of Rust being bigger. Node ships with an HTTP module, random number generation and regexes. Rust has neither of those. Maybe more functions in each modules?

by Zababa 5y ago

zozbot234 5y ago

It's ultimately Rust's reliance on generic code that forces us to deal with so many packages at build time. In idiomatic C/C++ much of the reusable code you compile ultimately turns into reusable shared objects, that sit in binary form on your fs with their corresponding include files. This is only possible in rare cases with Rust, because the "library" equivalents (crates) don't know how their generic types and code will be instantiated downstream. Everything is the equivalent of a "header only" lib.

geodel 5y ago

Yeah, in language design they might be on cutting edge. But in software engineering they want to be close to 90's, 00's cultish software design patterns, DRY principle and so on.

deaddabe 5y ago

Note that this separation is necessary if you want to achieve somewhat parallel compilation in Rust. Every proc_macro needs its own separate crate, so I end up providing two crates for macro-based stuff. At least, the stdlib seems to have enough batteries included for the dependency count to not be as high as NodeJS projects. From my experience I would rate the count of dependencies in this order: NodeJS > Rust > Python > C/C++ Though I cannot explain why this is ordered this way between {Node, Rust, Python}. Again, batteries included? Language popularity? Beginners-friendly programming language?

zozbot234 5y ago
AIUI, rustc now has ad-hoc parallelization of compile within a single crate. The defined compilation unit is still the crate though, as opposed to the single file in C/C++.
josephg 5y ago
> I cannot explain why this is ordered this way between {Node, Rust, Python}. Again, batteries included? My vote is that its cultural. Both for "batteries included" python, and for nodejs. Its easy to forget now, but npm was extremely innovative when it first came out. It absolutely went out of its way to make adding dependencies as easy as possible. And the culture (especially in the early days of nodejs) went bananas for this philosophy of programming. I had a chat with @isaacs on a bus one time. (He was the maintainer of nodejs at the time). I asked him what he thought about package documentation, and what to do when a README isn't enough. He said that he thinks if a readme isn't enough documentation for your library, your library is probably too big and should be split up. You still see this today with packages like "isobject" (a tiny function published as a package) which still gets 53M downloads / week[1]. Correct me if I'm wrong, but as I understand it python and ruby still don't support parallel dependencies with different versions in the build tree like npm does. If a python or ruby package transitively depends on foo@1.0 and foo@2.0 then my understanding is that ruby and python lose their minds. And this problem is almost impossible for the end user to solve. So libraries like rails sort of need to be designed as one big block of software. Nodejs has no problem with this - if you do this in nodejs, npm will just quietly install both versions and node will happily wire everything up correctly. The only constraint is that no single package can have a direct dependency on both foo@1 and foo@2 at the same time. But thats not something that you ever really want to do in practice. The tooling supported this too. It was quite common to have a library with some optional features that not many people used. If you wanted to exclude them from your javascript bundle, until recently the main way to do that was by breaking your library up into small pieces so your library's consumer could pick and choose what they wanted. Building a dependency tree out of with thousands of tiny modules is exactly what nodejs is designed for. It should come as no surprise that thats what we got. [1] https://www.npmjs.com/package/isobject https://www.npmjs.com/package/isobject
- zozbot234 5y ago
  Mentioned in a sibling comment already, but Rust has these features as well, and I think Go ended up getting them too although there was some controversy about the need for them.
Zababa 5y ago
I would argue that it's ordered by a mix between popularity, need for those packages and ease of installing packages. JS and Python are around the same order of magnitude of popularity, installing packages is way easier in JS than in Python (NPM might not be great but pip is hell). Rust is less popular but it's very easy to have lots of packages, plus what you mentioned about procedural macros. And the standard library, like JS, is relatively small. If you add Go to that (relatively easy to add package, less popular than Python and JS but more than Rust, lots of stuff in the standard library) which would be higher than C/C++ and lower than Python, it does seem to fit. I don't know anything about the C#/Java ecosystems, same with Ruby and Perl. I'd like to know if they fit this "model" too, that would be interesting. It could give some pointers on how to design/make a language evolve to avoid having lots of packages. I'm not sure about the standard library of Rust being bigger. Node ships with an HTTP module, random number generation and regexes. Rust has neither of those. Maybe more functions in each modules?