5 ms·

I don't think most of these are applicable to Rust. > - Conversions between our 5(!) string types are very common. > - Standard library I/O modules cannot use

I don't think most of these are applicable to Rust. > - Conversions between our 5(!) string types are very common. > - Standard library I/O modules cannot use new, de-facto standard string types (i.e. `Text` and `ByteString`) defined outside it because of dependency cycle. We have one string type defined in std, and nobody is defining new ones (modulo special cases for legacy encodings which would not be worth polluting the default string type with). > - Standard library cannot use containers, other than lists, for the same reason. > - No standard traits for containers, like maps and sets, as those are defined outside the standard library. Result is that code is written against one concrete implementation. Hash maps and trees are in the standard library already. Everyone uses them. > - Newtype wrapping to avoid orphan instances. Having traits defined in packages other than the standard library makes it harder to write non-orphan instances. This is true, but this hasn't been much of a problem in Rust thus far. > - It's too difficult to make larger changes as we cannot atomically update all the packages at once. Thus such changes don't happen. That only matters if you're breaking public APIs, right? That seems orthogonal to the small-versus-large-standard-library debate. Even if you have a large standard library, if you promised it's stable you still can't break APIs.

rtpg 10y ago
But if you have a large standard library and want to break the API, you can. If you have 100 different libs that are basically "standard" (who doesn't have `mtl` in their applications at this point), now you have to coordinate 100 different library updates roughly at the same time. If you forget even one of them, then you've broken everything. I think the argument for a large Prelude/standard lib is similar to Google's "single repo" argument: Easy to catch usages and fix them all at once. Plus you're making the language more useful out of the box. People coming from python can understand this feeling of opening a python shell and being productive super quickly form the get-go. Arguments for small std lib exist, of course. But Giant standard libraries are more useful than not. EDIT: I think the failure of the Haskell Platform has a lot more to do with how Haskell deals with dependencies, and the difficulties it entails, than with the "batteries included" approach itself.
- hyperhopper 10y ago
  But even then, languages that have great, thriving easy to use dependency systems and package managers with small standard libraries still run into problems see: javascript
  - wilmoore 10y ago
    The issue with comments written this way is that there are no details to support the claim. Writing "see: JavaScript" doesn't really help without context. Without context, one does no know if you meant "JavaScript in browser" or "JavaScript via Node.JS" or "I simply don't like npm". I'm not claiming there aren't any problems; however, "problems" are situational and one person's "problem" is another person's meh. I just think it's irresponsible to not provide detail when making such claims.
- pcwalton 10y ago
  > But if you have a large standard library and want to break the API, you can. We have a policy of no breakage for stable libraries post 1.0 (as does Python, and Go, etc.). So no, we can't. The size of the standard library has nothing to do with it.
- bshanks 10y ago
  I'm curious about which language features or tooling do other languages have that make them better at dealing with dependencies than Haskell?
- barrkel 10y ago
  Standard libraries - types, in particular - are the lingua franca between unrelated libraries. The more that's in your standard library, the easier it is to integrate different libraries. The higher level the library (e.g. containing content specific to an application domain), the more magical-seeming libraries can be added to the ecosystem. The counter-risk is the standard library growing in undesirable directions that you can never change because you can't remove stuff. The interstitial glue that lets third party libraries integrate with one another and be usable by your app: that's the single biggest reason for having a bigger standard library than a smaller one. It has very little to do with including the batteries in the box. If you think it has something to do with including the batteries in the box, you'll be lured into the trap of making it easy to fetch the batteries from across the internet (that's almost the same, right?). The trouble is, the internet has 100 different batteries to choose from, and not only have you offloaded the choice onto the user, but the batteries use mutually incompatible terminals and you have to jerry-rig interfaces between them. Let a thousand flowers bloom, say some people: trouble is, waiting for the biggest flower can take years, and people pick different ones in the early days. A bad choice is better than indecision. Low effort updates are even less what large standard libraries are about. Large standard libraries are much harder to update, not easier: there's much more surface area, so it's far easier to break an application - and since every application uses the standard library, you could potentially break them all. Easier versioning and updates are a strong argument for extracting out things into third-party libraries.
sitkack 10y ago
I think rust needs to slow down in this regard. I have been with Python since 1999 and the stdlib has held it back, I have also used Scala and Haskell and have witness the mess that platform libs on each have caused. What Rust has right now is pretty amazing. What needs to happen is a way for devs to easily break the dependency cycle and include multiple versions of the same crate. Something that has plagued Haskell. I dunno what the answer is, trait only crates, struct only crates? If people want to 'curate' (shop) a set of packages, they can make a meta package that exports its deps. There is literally no reason to ship libs with the compiler aside from the basic verbs and nouns. With verioned and properly name-spaced imports, one could use different curated libs.
- steveklabnik 10y ago
  Rust will already allow you to have multiple versions of transitive dependencies.
  - sitkack 10y ago
    This needs to be screamed from the hill tops!
  - bassislife 10y ago
    Which I guess is normal since it does not create any dependency cycle. A new version might as well be thought as a completely different package (of perhaps similar functionality).
  - wilmoore 10y ago
    To be honest, this is the kind of thing that would be great to highlight immediately on https://github.com/rust-lang/cargo https://github.com/rust-lang/cargo or http://doc.crates.io/guide.html http://doc.crates.io/guide.html. I go to those pages and I am given 0 information on how the thing actually works.
    - steveklabnik 10y ago
      Cargo's docs need a bunch of work, it's true. I have so much to do :(
legulere 10y ago
> We have one string type defined in std, and nobody is defining new ones (modulo special cases for legacy encodings which would not be worth polluting the default string type with). There's also `inlinable_string`, `string_cache`, `tendril`, `intern` if you need inlining for performance. The bigger problem is with other things like 2D/3D points which can be (f32, f32), [f32; 2] or a custom struct.
JoshTriplett 10y ago
> We have one string type defined in std The standard library also includes Path/PathBuf and OsStr/OsString. And third-party libraries also use [u8] for bytestrings. It'd be nice to improve handling for user-supplied text where you can't assume UTF-8. For instance, git2-rs provides the contents of diffs as [u8], because it can't assume the diffed files use UTF-8. That led to this commit today: https://github.com/ogham/rust-ansi-term/pull/19/commits/a0da28ad91c15cb90d85b79048b6a3878f4386e1 https://github.com/ogham/rust-ansi-term/pull/19/commits/a0da... That felt like a lot of boilerplate to abstract between str and [u8]. Is there a better way to solve that problem? (As much as I'd love to just say "use UTF-8", that would break on many git repositories, including git.git and linux.git.)
- Manishearth 10y ago
  > The standard library also includes Path/PathBuf and OsStr/OsString. Right. And you want people to explicitly convert between them. Having to convert between string types isn't a problem. String encoding is hard, and you're going to have to pay that cost somehow.
  - JoshTriplett 10y ago
    Having to convert isn't a problem. Having to write some algorithms multiple times for different string types is a problem.
    - Manishearth 10y ago
      Fair. Most of these algorithms could be written generically I guess.
- pcwalton 10y ago
  > That felt like a lot of boilerplate to abstract between str and [u8]. Is there a better way to solve that problem? This doesn't have anything to do with large vs. small standard libraries, because all of these string types are defined in libstd.
moomin 10y ago
I really really would advise having a word with Snoyberg about this. The Haskell Platform has been a pretty deadly experience. It's also ridiculously beginner-hostile (sounds like it won't be, is in practice).
barrkel 10y ago
Hash maps and trees: fine. What about database interfaces (e.g. a JDBC/ODBC/whatever equivalent)? What about HTTP servers - even the minimal declaration for what a synchronous request handler might look like? How about threadpooling - if you have multiple libraries that have parallelizable work, you certainly don't want multiple threadpools each thinking they have X many cores to work with, and you don't want the user to have to partition these things either - that's not a happy problem. All things you can delegate to third parties, but not without lots of cross-talk and confusion until things settle down to winners and losers, which may be a long time in the future. Indecision can be costly. Consider standard library profiles, with progressively higher levels of abstraction supported. It's the right decision for creating a good ecosystem. C and C++ took decades to build consensus on the more complicated libraries, and C++ eventually grew a pseudo-standard library in the form of boost to centralize efforts, simply because it is more efficient that way.

rtpg 10y ago

But if you have a large standard library and want to break the API, you can. If you have 100 different libs that are basically "standard" (who doesn't have `mtl` in their applications at this point), now you have to coordinate 100 different library updates roughly at the same time. If you forget even one of them, then you've broken everything. I think the argument for a large Prelude/standard lib is similar to Google's "single repo" argument: Easy to catch usages and fix them all at once. Plus you're making the language more useful out of the box. People coming from python can understand this feeling of opening a python shell and being productive super quickly form the get-go. Arguments for small std lib exist, of course. But Giant standard libraries are more useful than not. EDIT: I think the failure of the Haskell Platform has a lot more to do with how Haskell deals with dependencies, and the difficulties it entails, than with the "batteries included" approach itself.

hyperhopper 10y ago
But even then, languages that have great, thriving easy to use dependency systems and package managers with small standard libraries still run into problems see: javascript
- wilmoore 10y ago
  The issue with comments written this way is that there are no details to support the claim. Writing "see: JavaScript" doesn't really help without context. Without context, one does no know if you meant "JavaScript in browser" or "JavaScript via Node.JS" or "I simply don't like npm". I'm not claiming there aren't any problems; however, "problems" are situational and one person's "problem" is another person's meh. I just think it's irresponsible to not provide detail when making such claims.
pcwalton 10y ago
> But if you have a large standard library and want to break the API, you can. We have a policy of no breakage for stable libraries post 1.0 (as does Python, and Go, etc.). So no, we can't. The size of the standard library has nothing to do with it.
bshanks 10y ago
I'm curious about which language features or tooling do other languages have that make them better at dealing with dependencies than Haskell?
barrkel 10y ago
Standard libraries - types, in particular - are the lingua franca between unrelated libraries. The more that's in your standard library, the easier it is to integrate different libraries. The higher level the library (e.g. containing content specific to an application domain), the more magical-seeming libraries can be added to the ecosystem. The counter-risk is the standard library growing in undesirable directions that you can never change because you can't remove stuff. The interstitial glue that lets third party libraries integrate with one another and be usable by your app: that's the single biggest reason for having a bigger standard library than a smaller one. It has very little to do with including the batteries in the box. If you think it has something to do with including the batteries in the box, you'll be lured into the trap of making it easy to fetch the batteries from across the internet (that's almost the same, right?). The trouble is, the internet has 100 different batteries to choose from, and not only have you offloaded the choice onto the user, but the batteries use mutually incompatible terminals and you have to jerry-rig interfaces between them. Let a thousand flowers bloom, say some people: trouble is, waiting for the biggest flower can take years, and people pick different ones in the early days. A bad choice is better than indecision. Low effort updates are even less what large standard libraries are about. Large standard libraries are much harder to update, not easier: there's much more surface area, so it's far easier to break an application - and since every application uses the standard library, you could potentially break them all. Easier versioning and updates are a strong argument for extracting out things into third-party libraries.

sitkack 10y ago

I think rust needs to slow down in this regard. I have been with Python since 1999 and the stdlib has held it back, I have also used Scala and Haskell and have witness the mess that platform libs on each have caused. What Rust has right now is pretty amazing. What needs to happen is a way for devs to easily break the dependency cycle and include multiple versions of the same crate. Something that has plagued Haskell. I dunno what the answer is, trait only crates, struct only crates? If people want to 'curate' (shop) a set of packages, they can make a meta package that exports its deps. There is literally no reason to ship libs with the compiler aside from the basic verbs and nouns. With verioned and properly name-spaced imports, one could use different curated libs.

steveklabnik 10y ago
Rust will already allow you to have multiple versions of transitive dependencies.
- sitkack 10y ago
  This needs to be screamed from the hill tops!
- bassislife 10y ago
  Which I guess is normal since it does not create any dependency cycle. A new version might as well be thought as a completely different package (of perhaps similar functionality).
- wilmoore 10y ago
  To be honest, this is the kind of thing that would be great to highlight immediately on https://github.com/rust-lang/cargo https://github.com/rust-lang/cargo or http://doc.crates.io/guide.html http://doc.crates.io/guide.html. I go to those pages and I am given 0 information on how the thing actually works.
  - steveklabnik 10y ago
    Cargo's docs need a bunch of work, it's true. I have so much to do :(

legulere 10y ago

> We have one string type defined in std, and nobody is defining new ones (modulo special cases for legacy encodings which would not be worth polluting the default string type with). There's also `inlinable_string`, `string_cache`, `tendril`, `intern` if you need inlining for performance. The bigger problem is with other things like 2D/3D points which can be (f32, f32), [f32; 2] or a custom struct.

JoshTriplett 10y ago

> We have one string type defined in std The standard library also includes Path/PathBuf and OsStr/OsString. And third-party libraries also use [u8] for bytestrings. It'd be nice to improve handling for user-supplied text where you can't assume UTF-8. For instance, git2-rs provides the contents of diffs as [u8], because it can't assume the diffed files use UTF-8. That led to this commit today: https://github.com/ogham/rust-ansi-term/pull/19/commits/a0da28ad91c15cb90d85b79048b6a3878f4386e1 https://github.com/ogham/rust-ansi-term/pull/19/commits/a0da... That felt like a lot of boilerplate to abstract between str and [u8]. Is there a better way to solve that problem? (As much as I'd love to just say "use UTF-8", that would break on many git repositories, including git.git and linux.git.)

Manishearth 10y ago
> The standard library also includes Path/PathBuf and OsStr/OsString. Right. And you want people to explicitly convert between them. Having to convert between string types isn't a problem. String encoding is hard, and you're going to have to pay that cost somehow.
- JoshTriplett 10y ago
  Having to convert isn't a problem. Having to write some algorithms multiple times for different string types is a problem.
  - Manishearth 10y ago
    Fair. Most of these algorithms could be written generically I guess.
pcwalton 10y ago
> That felt like a lot of boilerplate to abstract between str and [u8]. Is there a better way to solve that problem? This doesn't have anything to do with large vs. small standard libraries, because all of these string types are defined in libstd.

moomin 10y ago

I really really would advise having a word with Snoyberg about this. The Haskell Platform has been a pretty deadly experience. It's also ridiculously beginner-hostile (sounds like it won't be, is in practice).

barrkel 10y ago

Hash maps and trees: fine. What about database interfaces (e.g. a JDBC/ODBC/whatever equivalent)? What about HTTP servers - even the minimal declaration for what a synchronous request handler might look like? How about threadpooling - if you have multiple libraries that have parallelizable work, you certainly don't want multiple threadpools each thinking they have X many cores to work with, and you don't want the user to have to partition these things either - that's not a happy problem. All things you can delegate to third parties, but not without lots of cross-talk and confusion until things settle down to winners and losers, which may be a long time in the future. Indecision can be costly. Consider standard library profiles, with progressively higher levels of abstraction supported. It's the right decision for creating a good ecosystem. C and C++ took decades to build consensus on the more complicated libraries, and C++ eventually grew a pseudo-standard library in the form of boost to centralize efforts, simply because it is more efficient that way.