13 ms·

At the risk of being slightly tangential, I've been sorely wanting to air this particular grievance with Rust for some time. It's somewhat related, since the au

by ajxs 7y ago

At the risk of being slightly tangential, I've been sorely wanting to air this particular grievance with Rust for some time. It's somewhat related, since the author mentions their package system. Its package ecosystem isn't nearly in the horrible state that node's is, but having a package system shouldn't be a substitute for designing a useful standard library for a language. I think that the attraction to 'small languages' is very much misplaced. If I can't get through Rust's official documentation without being recommended the use of third party packages for basic functionality (getopt, interfacing with static libraries... Etc) then the designers have made a terrible error.

Myrmornis 7y ago
Isn't it at least partly explained by wanting to allow implementations of this functionality to evolve more rapidly than they could if they were in the stdlib and thus reach good solutions more rapidly? getopt in particular seems like something that programmers haven't reached a consensus on despite 40 years of experimentation.
- ajxs 7y ago
  I would beg to differ with this. The kind of functionality expected from Getopt is fairly standard these days. I also disagree that features like getopt really need to 'evolve' much at this point in time. At the risk of courting controversy here, I'm going to confess that often I just don't want the community involved in the development of a language. On a project I inherited recently ( with the mission to save ) I unfortunately had to suffer using Typescript. If you look at the community discussions regarding the language's future direction you'll be treated to the uninformed arguing with the ignorant about the language's direction. Every time I download an external crate, I have to learn a different developer's way of doing things, suffer their idiosyncratic ways of creating an API and potentially expose myself or my application to a new set of vulnerabilities. The more I can get away with not doing this, the better.
  - sanderjd 7y ago
    Who is "the community" and who isn't? Are you sure you trust the core language developers more than people writing libraries?
    - ajxs 7y ago
      More often than not, absolutely. It requires a much higher level of competence to design a language and develop a functional compiler than it does to design libraries. There are packages in Node with millions of downloads, packages that are basically ubiquitous in certain domains that are riddled with bugs, with terrible interfaces and documentation. I can even think of libraries I've worked with in Java/C/.net/etc that are just horrific... If the languages themselves were as badly designed as the average library, they'd never succeed.
      - sanderjd 7y ago
        Language design and compiler implementation are definetely high-competency skills, but they don't necessarily correlate with library design skills. For instance, http client and server libraries are often in this gray area of uncertainty about whether they should be in the stdlib or not. Is this something language designers or compiler implementors have a lot of experience in? I would say not; sending and serving http requests are not something compilers need to do. Or take GUI libraries, what do language designers know about that? I also know of many bad third party libraries, but there are tons of examples of really awful parts of standard libraries. The original date/time APIs in Java are a mess (they finally fixed this by in essence bringing in a third party API). The ssl bindings in the Ruby stdlib were a common source of bugs back when I was paying attention to this (maybe they've fixed it), same thing for the built in http stuff. Someone else mentioned the similar weakness of python's built in http, such that most people use a third party library instead. Even the java collections APIs are pretty poor such that people often augment them with things like guava or apache libraries. My point is just that developing good libraries is a hard thing and I don't see any reason to think language designers or compiler maintainers are any better (or worse!) at it than other people. There isn't really a shortcut, you can't just cede authority to the powers that be on the core language teams, you just have to evaluate the quality of libraries for your use case yourself.
acheron9383 7y ago
The language is still 'fairly new'. I'd rather the lang team let popular things cook as a 3rd party package for awhile until a good default solution shakes out. For instance hashbrown was just pulled into the stdlib. Some people are taking a look at bringing Crossbeam into the stdlib. [1] https://internals.rust-lang.org/t/proposal-new-channels-for-rusts-standard-library/9548 https://internals.rust-lang.org/t/proposal-new-channels-for-...
- dralley 7y ago
  And some libraries that were included are now recognized to have been mistakes, e.g. std::sync::mpsc
burntsushi 7y ago
Opinions on this are a dime a dozen. You often see the reverse of it too, for example, you might have heard that "Python's standard library is where things go to die." You could just as easily call that a "terrible error." The fact that Python's standard library has an HTTP client in it, for example, doesn't stop everyone from using requests (and, consequently, urllib3) for all their HTTP client needs. So despite the fact the standard library provides a lot of the same functionality as a third party dependency, folks are still using the third party dependency. I think the size of the standard library is just one of possibly many contributing factors that leads to a large number of dependencies. I think a part of it is culture, but another part of it is that the tooling _enables_ it. It's so incredibly easy to write some code, push it to crates.io and let everyone else use it. That's generally a good thing, but it winds up creating this spiral where there's almost no backpressure _against_ including a dependency in a project. This means there's very little standing in the way of letting the fullest expression of DRY run wild. There are some notable examples in the NPM ecosystem where it reaches ridiculous levels. But putting the extremes aside, there's a ton of grey area and it can be pretty difficult to convince someone to write a bit more code when something else might work off the shelf. (And I mean this in the most charitable way possible. I find myself in that situation.) I do hope we can turn the Rust ecosystem around and stop regularly having dependency trees with hundreds of crates, but it's going to be a long and difficult road. For example, not everyone even agrees with my perspective that this is actually a bad thing.
- Amezarak 7y ago
  There are a lot of us out there with dev and prod environments that aren’t allowed to be connected to the Internet or otherwise strictly controlled. Languages that rely on me downloading packages are mostly dead in the water at my workplace.
  - steveklabnik 7y ago
    Firefox, Debian, and many other build systems aren’t either. That’s orthogonal from all of this; the use case is well supported and has been for years.
  - 7y ago
vvanders 7y ago
On the flip side Lua is a wonderful language because its small lib lets it go anywhere. We used to run a whole gamestate of a shipped title on PSP in a 400kb block allocation. I've yet to see that in any other dynamic language of consequence.
- hinkley 7y ago
  When I was still in that 'I should write a programming language' stage of career development, I worked on a pretty sophisticated (for the era) mobile app. PyPy was getting quite a bit of press around that time and my brain connected some dots. One of the ideas I wanted very much to explore is scaling the API, both up and down. For building something akin to PyPy, you might want a 'kernel', a small set of libraries that were available everywhere, and several other levels that include more or different things. Mobile takes the base and adds a few things suitable for mobile (storage, UI, broader networking). Desktop has a real UI, and then there's the kitchen sink like .Net and Java have. But you have the same problems you always have with decomposition - if you didn't guess the right boundaries when you built the thing then removing or rearranging bits is a serious PITA. Sometimes I think the best we can hope for is to leave clear messages for the next language so that it doesn't organize things the way we did.
- nonbirithm 7y ago
  LuaJIT is incredibly fast. It might just be the fastest scripting language out there. I think this was only possible because Lua itself is such a simple language. It says a lot that LuaJIT's implementation of this simple language is extremely complex in comparison to vanilla 5.1. Imagine the added complexity for something like Ruby that offers more than one associative data structure. With LOVE you get a complete game engine runtime with all the boring OS abstractions taken care of and you can just start working. I tried a few game development bouts with Rust since it seemed like a natural step up from C++ but the compile cycle kills a lot of my creative drive. I think this is just a consequence of the language being compiled, as with C++. With Rust having a wide variety of language features the compile time is increased also. At least we're lucky to have languages today that offer far shorter turnaround times for building prototypes.
- pitaj 7y ago
hinkley 7y ago
> Its package ecosystem isn't nearly in the horrible state that node's is I propose that the dictionary entry for 'faint praise' be updated to include this example.
spieglt 7y ago
If Rust had a standard library like Go's I'd love it even more.
- pcwalton 7y ago
  Even if that library had slow serialization like that of Go, forcing the use of packages to make it faster?
  - ninkendo 7y ago
    So we'd be no worse off than we are now? And just in that one case?
    - TheCoelacanth 7y ago
      Except for all of the wasted work for maintainers to maintain the crappy library that has a better replacement outside of the standard library.
- majewsky 7y ago
  Go's standard library works for Go because it has a rather sharp focus on implementing web services. It's worthless for implementing a GUI application, or a particle physics simulation, or a PID-1 daemon. Rust has a much broader aim, so a stdlib accommodating all of its usecases would be as comically huge as Python's, with all the problems that come from that.
  - deleted 7y ago
    [deleted]
  - spieglt 7y ago
    It's also good for crypto, image processing, logging, file compression, and other commonly useful things. Go does have an emphasis on network services but is not limited to that. "a stdlib accommodating all of its usecases would be as comically huge as Python's, with all the problems that come from that." That's a straw man. I don't want a stdlib like Python's, I want one like Go's.
falcolas 7y ago
One very quick comment in support of a strong first party library: it’s proven to be much simpler, and much more common, for third party libraries to be compromised and everything from intentional security holes to full on malware to be bundled into the distributed packages. I could pin every dependency, and monitor CVEs for all my dependencies, and monitor the ownership/code changes for all my dependencies, and hope for updates in a timely manner should there be CVEs... or I can use the standard library and move on with my life. Personal opinion of course. Some people enjoy monitoring CVE lists. :)
pcwalton 7y ago
Rust used to have getopt in the standard library. It was removed because it was bad. Clap only came later. I don't think Rust would have been a better language if the old getopt were kept around.
giancarlostoro 7y ago
> but having a package system shouldn't be a substitute for designing a useful standard library for a language. Completely agree. I hold firm to the believe that a strong standard library that considers modern software development goals will drive the success of a language. Look at Go. Which is usually my example since not many do what Go does. You can do a whole web application in Go with minimal use (if any) of external libraries. Back to Rust and their defense: they intend to adopt popular / quality packages from Cargo into the standard library or fill the gap between packages. I secretly wish D had similar things to Go in the standard library. HTTP and SMTP etc which Python seems to have at least HTTP and sadly Go ditched their SMTP package for whatever reason making it awkward. I hate having to learn a new package manager per new language. I love programming so I try a lot of languages out of love. Package managers and build systems are horrible UX in every language. I prefer to not rely on third party oh look now deprecated packages and start out with whats out of the box.
roca 7y ago
This issue gets discussed over and over again. Rust does need a better package curation story, a Rust "expanded universe" of recommended packages whose provenance is more carefully tracked than the average package and whose APIs are stable. The Rust community needs to encourage other packages to depend on stable versions in that recommended set to reduce duplication. But going further and actually making that set to be part of the "standard library", and thus released at the cadence of the Rust language itself, and managed under the same umbrella, would be harmful.
majewsky 7y ago
My main gripe with crates.io is that they allowed everyone to take every name. The "foo" crate just belongs to the first person who took (or squatted) that name, regardless of whether that person actually implements a nice library under that name. What they should've done is allow uploads only to "<username>/<cratename>". So if I decide to make a regex crate, it's called "majewsky/regex" at first and Alice can make "alice/regex" and Bob can make "bob/regex". Then at some point, the community (through some open process) decides that Alice's crate has the best API, so "alice/regex" gets aliased to just "regex". That way, everything in the main namespace adheres to some sort of quality standard and has community support behind it. Because there's some explicit process gate to getting stuff into the main namespace, you could attach any number of beneficial requirements to it, e.g.: - test coverage - documentation coverage - at least 3 people having committer rights to the repo, at least 2 of which must not be affiliated with the same company
- fnord123 7y ago
  Apparently that was done in the Ruby community and Github "helpfully" uploaded all the Ruby packages to the gems repository meaning all 217 forks of https://github.com/httprb/http https://github.com/httprb/http became potential gems. But the underlying idea is sound: namespace and federation. Rust should ideally follow Java's solution. And it's completely forward portable: when namespacing is introduced, the crates.io/regex crates become io.crates.regex. Then you can have your com.github.majewsky.regex.
  - majewsky 7y ago
    I don't particularly like URL-derived package names, esp. with Go where the repo URL is auto-derived from the package name, because that makes it an absolute pita to move the repo to a new canonical location.
    - fnord123 7y ago
      That's not an issue in Java.
- 7y ago
deleted 7y ago
[deleted]
baby 7y ago
This, Rust std library is ridiculous compared to Golang’s one.
ameixaseca 7y ago
Given there are no infinite resources, one has to choose where to focus. Rust team seems to be focusing on the language and solving hard-to-deal-with but relevant problems - and on evolving the language, which is truly a remarkable feat when you are pushing the boundaries. Libraries can be implemented by 3rd parties and maybe later adopted as standard libraries or become de-facto standards. That's how it has been done for most popular languages out there, and I think it's a smart decision.

by ajxs 7y ago

Myrmornis 7y ago

Isn't it at least partly explained by wanting to allow implementations of this functionality to evolve more rapidly than they could if they were in the stdlib and thus reach good solutions more rapidly? getopt in particular seems like something that programmers haven't reached a consensus on despite 40 years of experimentation.

ajxs 7y ago
I would beg to differ with this. The kind of functionality expected from Getopt is fairly standard these days. I also disagree that features like getopt really need to 'evolve' much at this point in time. At the risk of courting controversy here, I'm going to confess that often I just don't want the community involved in the development of a language. On a project I inherited recently ( with the mission to save ) I unfortunately had to suffer using Typescript. If you look at the community discussions regarding the language's future direction you'll be treated to the uninformed arguing with the ignorant about the language's direction. Every time I download an external crate, I have to learn a different developer's way of doing things, suffer their idiosyncratic ways of creating an API and potentially expose myself or my application to a new set of vulnerabilities. The more I can get away with not doing this, the better.
- sanderjd 7y ago
  Who is "the community" and who isn't? Are you sure you trust the core language developers more than people writing libraries?
  - ajxs 7y ago
    More often than not, absolutely. It requires a much higher level of competence to design a language and develop a functional compiler than it does to design libraries. There are packages in Node with millions of downloads, packages that are basically ubiquitous in certain domains that are riddled with bugs, with terrible interfaces and documentation. I can even think of libraries I've worked with in Java/C/.net/etc that are just horrific... If the languages themselves were as badly designed as the average library, they'd never succeed.
    - sanderjd 7y ago
      Language design and compiler implementation are definetely high-competency skills, but they don't necessarily correlate with library design skills. For instance, http client and server libraries are often in this gray area of uncertainty about whether they should be in the stdlib or not. Is this something language designers or compiler implementors have a lot of experience in? I would say not; sending and serving http requests are not something compilers need to do. Or take GUI libraries, what do language designers know about that? I also know of many bad third party libraries, but there are tons of examples of really awful parts of standard libraries. The original date/time APIs in Java are a mess (they finally fixed this by in essence bringing in a third party API). The ssl bindings in the Ruby stdlib were a common source of bugs back when I was paying attention to this (maybe they've fixed it), same thing for the built in http stuff. Someone else mentioned the similar weakness of python's built in http, such that most people use a third party library instead. Even the java collections APIs are pretty poor such that people often augment them with things like guava or apache libraries. My point is just that developing good libraries is a hard thing and I don't see any reason to think language designers or compiler maintainers are any better (or worse!) at it than other people. There isn't really a shortcut, you can't just cede authority to the powers that be on the core language teams, you just have to evaluate the quality of libraries for your use case yourself.

acheron9383 7y ago

The language is still 'fairly new'. I'd rather the lang team let popular things cook as a 3rd party package for awhile until a good default solution shakes out. For instance hashbrown was just pulled into the stdlib. Some people are taking a look at bringing Crossbeam into the stdlib. [1] https://internals.rust-lang.org/t/proposal-new-channels-for-rusts-standard-library/9548 https://internals.rust-lang.org/t/proposal-new-channels-for-...

dralley 7y ago
And some libraries that were included are now recognized to have been mistakes, e.g. std::sync::mpsc

burntsushi 7y ago

Opinions on this are a dime a dozen. You often see the reverse of it too, for example, you might have heard that "Python's standard library is where things go to die." You could just as easily call that a "terrible error." The fact that Python's standard library has an HTTP client in it, for example, doesn't stop everyone from using requests (and, consequently, urllib3) for all their HTTP client needs. So despite the fact the standard library provides a lot of the same functionality as a third party dependency, folks are still using the third party dependency. I think the size of the standard library is just one of possibly many contributing factors that leads to a large number of dependencies. I think a part of it is culture, but another part of it is that the tooling _enables_ it. It's so incredibly easy to write some code, push it to crates.io and let everyone else use it. That's generally a good thing, but it winds up creating this spiral where there's almost no backpressure _against_ including a dependency in a project. This means there's very little standing in the way of letting the fullest expression of DRY run wild. There are some notable examples in the NPM ecosystem where it reaches ridiculous levels. But putting the extremes aside, there's a ton of grey area and it can be pretty difficult to convince someone to write a bit more code when something else might work off the shelf. (And I mean this in the most charitable way possible. I find myself in that situation.) I do hope we can turn the Rust ecosystem around and stop regularly having dependency trees with hundreds of crates, but it's going to be a long and difficult road. For example, not everyone even agrees with my perspective that this is actually a bad thing.

Amezarak 7y ago
There are a lot of us out there with dev and prod environments that aren’t allowed to be connected to the Internet or otherwise strictly controlled. Languages that rely on me downloading packages are mostly dead in the water at my workplace.
- steveklabnik 7y ago
  Firefox, Debian, and many other build systems aren’t either. That’s orthogonal from all of this; the use case is well supported and has been for years.
- 7y ago

vvanders 7y ago

On the flip side Lua is a wonderful language because its small lib lets it go anywhere. We used to run a whole gamestate of a shipped title on PSP in a 400kb block allocation. I've yet to see that in any other dynamic language of consequence.

hinkley 7y ago
When I was still in that 'I should write a programming language' stage of career development, I worked on a pretty sophisticated (for the era) mobile app. PyPy was getting quite a bit of press around that time and my brain connected some dots. One of the ideas I wanted very much to explore is scaling the API, both up and down. For building something akin to PyPy, you might want a 'kernel', a small set of libraries that were available everywhere, and several other levels that include more or different things. Mobile takes the base and adds a few things suitable for mobile (storage, UI, broader networking). Desktop has a real UI, and then there's the kitchen sink like .Net and Java have. But you have the same problems you always have with decomposition - if you didn't guess the right boundaries when you built the thing then removing or rearranging bits is a serious PITA. Sometimes I think the best we can hope for is to leave clear messages for the next language so that it doesn't organize things the way we did.
nonbirithm 7y ago
LuaJIT is incredibly fast. It might just be the fastest scripting language out there. I think this was only possible because Lua itself is such a simple language. It says a lot that LuaJIT's implementation of this simple language is extremely complex in comparison to vanilla 5.1. Imagine the added complexity for something like Ruby that offers more than one associative data structure. With LOVE you get a complete game engine runtime with all the boring OS abstractions taken care of and you can just start working. I tried a few game development bouts with Rust since it seemed like a natural step up from C++ but the compile cycle kills a lot of my creative drive. I think this is just a consequence of the language being compiled, as with C++. With Rust having a wide variety of language features the compile time is increased also. At least we're lucky to have languages today that offer far shorter turnaround times for building prototypes.
pitaj 7y ago

hinkley 7y ago

> Its package ecosystem isn't nearly in the horrible state that node's is I propose that the dictionary entry for 'faint praise' be updated to include this example.

spieglt 7y ago

If Rust had a standard library like Go's I'd love it even more.

pcwalton 7y ago
Even if that library had slow serialization like that of Go, forcing the use of packages to make it faster?
- ninkendo 7y ago
  So we'd be no worse off than we are now? And just in that one case?
  - TheCoelacanth 7y ago
    Except for all of the wasted work for maintainers to maintain the crappy library that has a better replacement outside of the standard library.
majewsky 7y ago
Go's standard library works for Go because it has a rather sharp focus on implementing web services. It's worthless for implementing a GUI application, or a particle physics simulation, or a PID-1 daemon. Rust has a much broader aim, so a stdlib accommodating all of its usecases would be as comically huge as Python's, with all the problems that come from that.
- deleted 7y ago
  [deleted]
- spieglt 7y ago
  It's also good for crypto, image processing, logging, file compression, and other commonly useful things. Go does have an emphasis on network services but is not limited to that. "a stdlib accommodating all of its usecases would be as comically huge as Python's, with all the problems that come from that." That's a straw man. I don't want a stdlib like Python's, I want one like Go's.

falcolas 7y ago

One very quick comment in support of a strong first party library: it’s proven to be much simpler, and much more common, for third party libraries to be compromised and everything from intentional security holes to full on malware to be bundled into the distributed packages. I could pin every dependency, and monitor CVEs for all my dependencies, and monitor the ownership/code changes for all my dependencies, and hope for updates in a timely manner should there be CVEs... or I can use the standard library and move on with my life. Personal opinion of course. Some people enjoy monitoring CVE lists. :)

pcwalton 7y ago

Rust used to have getopt in the standard library. It was removed because it was bad. Clap only came later. I don't think Rust would have been a better language if the old getopt were kept around.

giancarlostoro 7y ago

> but having a package system shouldn't be a substitute for designing a useful standard library for a language. Completely agree. I hold firm to the believe that a strong standard library that considers modern software development goals will drive the success of a language. Look at Go. Which is usually my example since not many do what Go does. You can do a whole web application in Go with minimal use (if any) of external libraries. Back to Rust and their defense: they intend to adopt popular / quality packages from Cargo into the standard library or fill the gap between packages. I secretly wish D had similar things to Go in the standard library. HTTP and SMTP etc which Python seems to have at least HTTP and sadly Go ditched their SMTP package for whatever reason making it awkward. I hate having to learn a new package manager per new language. I love programming so I try a lot of languages out of love. Package managers and build systems are horrible UX in every language. I prefer to not rely on third party oh look now deprecated packages and start out with whats out of the box.

roca 7y ago

This issue gets discussed over and over again. Rust does need a better package curation story, a Rust "expanded universe" of recommended packages whose provenance is more carefully tracked than the average package and whose APIs are stable. The Rust community needs to encourage other packages to depend on stable versions in that recommended set to reduce duplication. But going further and actually making that set to be part of the "standard library", and thus released at the cadence of the Rust language itself, and managed under the same umbrella, would be harmful.

majewsky 7y ago

My main gripe with crates.io is that they allowed everyone to take every name. The "foo" crate just belongs to the first person who took (or squatted) that name, regardless of whether that person actually implements a nice library under that name. What they should've done is allow uploads only to "<username>/<cratename>". So if I decide to make a regex crate, it's called "majewsky/regex" at first and Alice can make "alice/regex" and Bob can make "bob/regex". Then at some point, the community (through some open process) decides that Alice's crate has the best API, so "alice/regex" gets aliased to just "regex". That way, everything in the main namespace adheres to some sort of quality standard and has community support behind it. Because there's some explicit process gate to getting stuff into the main namespace, you could attach any number of beneficial requirements to it, e.g.: - test coverage - documentation coverage - at least 3 people having committer rights to the repo, at least 2 of which must not be affiliated with the same company

fnord123 7y ago
Apparently that was done in the Ruby community and Github "helpfully" uploaded all the Ruby packages to the gems repository meaning all 217 forks of https://github.com/httprb/http https://github.com/httprb/http became potential gems. But the underlying idea is sound: namespace and federation. Rust should ideally follow Java's solution. And it's completely forward portable: when namespacing is introduced, the crates.io/regex crates become io.crates.regex. Then you can have your com.github.majewsky.regex.
- majewsky 7y ago
  I don't particularly like URL-derived package names, esp. with Go where the repo URL is auto-derived from the package name, because that makes it an absolute pita to move the repo to a new canonical location.
  - fnord123 7y ago
    That's not an issue in Java.
7y ago

deleted 7y ago

[deleted]

baby 7y ago

This, Rust std library is ridiculous compared to Golang’s one.

ameixaseca 7y ago

Given there are no infinite resources, one has to choose where to focus. Rust team seems to be focusing on the language and solving hard-to-deal-with but relevant problems - and on evolving the language, which is truly a remarkable feat when you are pushing the boundaries. Libraries can be implemented by 3rd parties and maybe later adopted as standard libraries or become de-facto standards. That's how it has been done for most popular languages out there, and I think it's a smart decision.