4 ms·

Coming from Python and having no Rust experience, it's odd to me that these are all external packages and not apart of the standard library. For instance, Pyth

Coming from Python and having no Rust experience, it's odd to me that these are all external packages and not apart of the standard library. For instance, Python provides HTTP, regex, logging, etc. out of the box. Is this a difference in philosophy between Python and Rust? (Batteries-included vs unix?)

ekidd 6y ago
There are a couple of key differences between the two ecosystems here which affect the tradeoffs: 1. Rust's cargo tool manages dependencies, it's almost universally used, and it does a good job. It's most similar to a mature Pipenv. 2. The Rust community normally does a good job with semver and the type checker makes it much easier to notice breaking API changes. 3. The Rust standard library has a strict policy of not breaking backwards compatibility. 4. There are tools like cargo-audit, cargo-geiger, cargo-crev and cargo-deny that can be used to enforce various policies across all dependencies. 5. Rust allows building single-file executables, so you don't need to distribute dependencies at runtime. So in practice, people tend to standardize one or two good solutions outside of the standard library, and very occasionally the best ideas will make it into the standard library. It's partly a philosophical difference, but the entire ecosystem of tools works together to support that philosophy.
raphlinus 6y ago
Yes, it is a difference in philosophy. Here is a blog post from 3 years ago where the libs team explored making an official set of libs to more closely match a "batteries included" experience: https://blog.rust-lang.org/2017/05/05/libz-blitz.html https://blog.rust-lang.org/2017/05/05/libz-blitz.html This effort did not end up succeeding. Instead, a more organic, informal process seems to be working pretty well. Lists like this seem to be useful, even if none is officially blessed.
tlamponi 6y ago
A few "two cents"/opinions from me, surely not the complete picture. External crates are easy to include and often come also from rust core developers, this does not actually slows one down but allows for much more flexibility. The big benefit over python is that rust does not dictates what you need to use. And with the trait system you can often just plug in another, now maybe better fitting, solution for a specific crate. Further, note that rust is normally shipped to the user compiled to a binary with all rust code statically linked in, while python isn't. Python is shipped often as code, and if the std library would not include that much that'd mean that one had to ship lots of module code along. So, rust would not actually profit from having a big standard library the same way Python does. Having a big standard library makes it also harder to deprecate and remove cruft (<https://lwn.net/Articles/790677/ https://lwn.net/Articles/790677/>), which can get a problem over the long time.
mikepurvis 6y ago
Not a rustacean myself, but I think the key issue is that Python the language predates PyPI and the whole modern mentality of it being trivial to pull in lots of third party stuff, so the expectation was that the standard library needed to be "batteries included" for the language's big initial use-cases. So part of it is that Crates has always been a first-class part of the Rust ecosystem, but the other piece is observing from Python what happens when some key functionality in the stdlib ends up completely stagnated because it can only ever add features or fix bugs with a new language release (urllib/urllib2/urllib3 vs requests, asyncio vs trio, etc). So Rust prioritizes stabilizing the kinds of interfaces and language features which allow third party functionality to plug together in a reasonable way, rather than committing to supplying all of in the core.
baq 6y ago
stdlib of any language is a place where libraries go to die. i agree it's surprising but it makes sense - cargo works very well and some packages are created by the same team that develops the stdlib, except they're free to break the api once in a while as needed.
- pjmlp 6y ago
  The stdlib is where libraries guaranteed to work across all platforms that the language targets also exist.
  - Ar-Curunir 6y ago
    That’s not true; the Rust standard library contains things that are OS/architecture specific. (Anyway, in rust cross target compatibility is usually not a worry if you’re not using unsafe, a dependency that uses C libraries)
    - pjmlp 6y ago
      All standard libraries contain OS specific code for their implementation.
  - ChrisSD 6y ago
    In Rust this is only guaranteed to be true of "tier 1" targets. Rust also supports many "tier 2" and lower targets where the guarantee is only that the standard library will build (not that it actually works) and even then on some platforms only the core is supported.
  - baq 6y ago
    yeah that was the point - less guarantees that something works forever means less maintenance. in the current world the balance is with leaner stdlibs because discovery and acquisition of external libraries is so simple.
rkangel 6y ago
Yes, it is a purposeful difference in philisophy. Python has challenges from the fact that its standard library is big - the interfaces (and so design) for a lot of these things are locked in place and you get stuck with suboptimal situations. And a lot of maintenance is required by the core team. Python 3 used the opportunity of backwards incompatibility to fix the worst issues, but that's a one time opportunity. Rust consciously decided to keep the standard library minimal due to the backward compatibility guarantees they want to give, and then provide an excellent packaging story so that it's easy to pull in more capabilities. This allows for much faster iteration and improvement on what Python might consider 'core' capabilities. Lots of the modules provided here are not the first attempt to achieve what they're doing. They're in the list because (the author considers them) the best. If they were in the standard library, maybe we'd be stuck with a worse version?
mjw1007 6y ago
In addition to the issues other people have mentioned, Python benefits from a large standard library in a way that Rust doesn't, because Python scripts don't require a build step. So it's easier to use Python than Rust for a little bit of "glue" in a place where maintaining a pile of dependencies would be inconvenient. If you can just stick a single .py file in a git repo somewhere and arrange for it to be run, that's a big gain compared to having to maintain a list of dependencies and arrange for them to be installed and so on. So it makes a big difference that you have access to regular expressions and tar files and whatever in that situation.
luizfelberti 6y ago
This has been one of the biggest pain points for me when using Rust, and I think this is a bad philosophy. Leaving the implementation of basic functionality, that everyone needs in any language if you're trying to do anything even slightly non-trivial, to the "organic process of letting things naturally arise from the community", has so far been a very effective way to ensure no packages can naturally talk to each other ever, or have any composability. Everyone re-implements their own "personal abstractions library of crap I need often" like we used to do with C because package management in C is horrid and the stdlib is as crude as it comes, or use one of the many crates of people who decided to publish their 10 LOC crate, and then you get a proliferation of microscopic packages that implement trivial functionality and the ecosystem starts looking a lot like NPM. Rust could really use some more top-down leadership. Say what you want about Rob Pike (and I'm not saying that Golang doesn't have pain-points with the leadership), but almost everything in the ecosystem composes like magic. On top of that, everytime I try to use Rust I'll need a feature and will run into some issue from 8 years ago, where everybody admits it's a problem, there are good and obvious solutions on the table, but all those 8 years were spent circlejerking and bikeshedding and no one can make up their minds; it's not even a "this is what the future of this issue looks like but it will take some time", it's more like a "after 8 years consensus hasn't been reached and no progress has been made on something a lot of people agree is a problem". Sometimes the need for group consensus serves only to corrupt the conceptual integrity of systems, and dilute any sense of vision into oblivion... I wish we could do without it sometimes
- Ar-Curunir 6y ago
  Lol, some people complain about too many dependencies, some people complain about dependencies being shit, you can never win
  - colejohnson66 6y ago
    Yep. If a language has too small a standard library, you end up with NPM for JavaScript. Too big a library and you end up with C++. Can’t please everyone...
pornel 6y ago
Batteries included are becoming a problem for Python: https://pyfound.blogspot.com/2019/05/amber-brown-batteries-included-but.html https://pyfound.blogspot.com/2019/05/amber-brown-batteries-i... Rust promises to never break or remove anything from the standard library, and Rust also wants to be a language usable for the next 40 years. This means that it has to be careful not to add stuff that's going to become outdated and deprecated cruft that they'll have to maintain forever. Rust has already made a mistake of including a channel implementation in stdlib, which ended up being inferior to crossbeam-channel. Now, as you can even see from TFA, there's a dilemma whether to use the slower, less flexible one from stdlib, or a crate. If there's only a crate, there's no dilemma :)

ekidd 6y ago

There are a couple of key differences between the two ecosystems here which affect the tradeoffs: 1. Rust's cargo tool manages dependencies, it's almost universally used, and it does a good job. It's most similar to a mature Pipenv. 2. The Rust community normally does a good job with semver and the type checker makes it much easier to notice breaking API changes. 3. The Rust standard library has a strict policy of not breaking backwards compatibility. 4. There are tools like cargo-audit, cargo-geiger, cargo-crev and cargo-deny that can be used to enforce various policies across all dependencies. 5. Rust allows building single-file executables, so you don't need to distribute dependencies at runtime. So in practice, people tend to standardize one or two good solutions outside of the standard library, and very occasionally the best ideas will make it into the standard library. It's partly a philosophical difference, but the entire ecosystem of tools works together to support that philosophy.

raphlinus 6y ago

Yes, it is a difference in philosophy. Here is a blog post from 3 years ago where the libs team explored making an official set of libs to more closely match a "batteries included" experience: https://blog.rust-lang.org/2017/05/05/libz-blitz.html https://blog.rust-lang.org/2017/05/05/libz-blitz.html This effort did not end up succeeding. Instead, a more organic, informal process seems to be working pretty well. Lists like this seem to be useful, even if none is officially blessed.

tlamponi 6y ago

A few "two cents"/opinions from me, surely not the complete picture. External crates are easy to include and often come also from rust core developers, this does not actually slows one down but allows for much more flexibility. The big benefit over python is that rust does not dictates what you need to use. And with the trait system you can often just plug in another, now maybe better fitting, solution for a specific crate. Further, note that rust is normally shipped to the user compiled to a binary with all rust code statically linked in, while python isn't. Python is shipped often as code, and if the std library would not include that much that'd mean that one had to ship lots of module code along. So, rust would not actually profit from having a big standard library the same way Python does. Having a big standard library makes it also harder to deprecate and remove cruft (<https://lwn.net/Articles/790677/ https://lwn.net/Articles/790677/>), which can get a problem over the long time.

mikepurvis 6y ago

Not a rustacean myself, but I think the key issue is that Python the language predates PyPI and the whole modern mentality of it being trivial to pull in lots of third party stuff, so the expectation was that the standard library needed to be "batteries included" for the language's big initial use-cases. So part of it is that Crates has always been a first-class part of the Rust ecosystem, but the other piece is observing from Python what happens when some key functionality in the stdlib ends up completely stagnated because it can only ever add features or fix bugs with a new language release (urllib/urllib2/urllib3 vs requests, asyncio vs trio, etc). So Rust prioritizes stabilizing the kinds of interfaces and language features which allow third party functionality to plug together in a reasonable way, rather than committing to supplying all of in the core.

baq 6y ago

stdlib of any language is a place where libraries go to die. i agree it's surprising but it makes sense - cargo works very well and some packages are created by the same team that develops the stdlib, except they're free to break the api once in a while as needed.

pjmlp 6y ago
The stdlib is where libraries guaranteed to work across all platforms that the language targets also exist.
- Ar-Curunir 6y ago
  That’s not true; the Rust standard library contains things that are OS/architecture specific. (Anyway, in rust cross target compatibility is usually not a worry if you’re not using unsafe, a dependency that uses C libraries)
  - pjmlp 6y ago
    All standard libraries contain OS specific code for their implementation.
- ChrisSD 6y ago
  In Rust this is only guaranteed to be true of "tier 1" targets. Rust also supports many "tier 2" and lower targets where the guarantee is only that the standard library will build (not that it actually works) and even then on some platforms only the core is supported.
- baq 6y ago
  yeah that was the point - less guarantees that something works forever means less maintenance. in the current world the balance is with leaner stdlibs because discovery and acquisition of external libraries is so simple.

rkangel 6y ago

Yes, it is a purposeful difference in philisophy. Python has challenges from the fact that its standard library is big - the interfaces (and so design) for a lot of these things are locked in place and you get stuck with suboptimal situations. And a lot of maintenance is required by the core team. Python 3 used the opportunity of backwards incompatibility to fix the worst issues, but that's a one time opportunity. Rust consciously decided to keep the standard library minimal due to the backward compatibility guarantees they want to give, and then provide an excellent packaging story so that it's easy to pull in more capabilities. This allows for much faster iteration and improvement on what Python might consider 'core' capabilities. Lots of the modules provided here are not the first attempt to achieve what they're doing. They're in the list because (the author considers them) the best. If they were in the standard library, maybe we'd be stuck with a worse version?

mjw1007 6y ago

In addition to the issues other people have mentioned, Python benefits from a large standard library in a way that Rust doesn't, because Python scripts don't require a build step. So it's easier to use Python than Rust for a little bit of "glue" in a place where maintaining a pile of dependencies would be inconvenient. If you can just stick a single .py file in a git repo somewhere and arrange for it to be run, that's a big gain compared to having to maintain a list of dependencies and arrange for them to be installed and so on. So it makes a big difference that you have access to regular expressions and tar files and whatever in that situation.

luizfelberti 6y ago

This has been one of the biggest pain points for me when using Rust, and I think this is a bad philosophy. Leaving the implementation of basic functionality, that everyone needs in any language if you're trying to do anything even slightly non-trivial, to the "organic process of letting things naturally arise from the community", has so far been a very effective way to ensure no packages can naturally talk to each other ever, or have any composability. Everyone re-implements their own "personal abstractions library of crap I need often" like we used to do with C because package management in C is horrid and the stdlib is as crude as it comes, or use one of the many crates of people who decided to publish their 10 LOC crate, and then you get a proliferation of microscopic packages that implement trivial functionality and the ecosystem starts looking a lot like NPM. Rust could really use some more top-down leadership. Say what you want about Rob Pike (and I'm not saying that Golang doesn't have pain-points with the leadership), but almost everything in the ecosystem composes like magic. On top of that, everytime I try to use Rust I'll need a feature and will run into some issue from 8 years ago, where everybody admits it's a problem, there are good and obvious solutions on the table, but all those 8 years were spent circlejerking and bikeshedding and no one can make up their minds; it's not even a "this is what the future of this issue looks like but it will take some time", it's more like a "after 8 years consensus hasn't been reached and no progress has been made on something a lot of people agree is a problem". Sometimes the need for group consensus serves only to corrupt the conceptual integrity of systems, and dilute any sense of vision into oblivion... I wish we could do without it sometimes

Ar-Curunir 6y ago
Lol, some people complain about too many dependencies, some people complain about dependencies being shit, you can never win
- colejohnson66 6y ago
  Yep. If a language has too small a standard library, you end up with NPM for JavaScript. Too big a library and you end up with C++. Can’t please everyone...

pornel 6y ago

Batteries included are becoming a problem for Python: https://pyfound.blogspot.com/2019/05/amber-brown-batteries-included-but.html https://pyfound.blogspot.com/2019/05/amber-brown-batteries-i... Rust promises to never break or remove anything from the standard library, and Rust also wants to be a language usable for the next 40 years. This means that it has to be careful not to add stuff that's going to become outdated and deprecated cruft that they'll have to maintain forever. Rust has already made a mistake of including a channel implementation in stdlib, which ended up being inferior to crossbeam-channel. Now, as you can even see from TFA, there's a dilemma whether to use the slower, less flexible one from stdlib, or a crate. If there's only a crate, there's no dilemma :)