4 ms·

I'm curious if rust has this problem. The problem I notice in npm land is many developers have no taste. Example, there's a library for globbing call glob. You'

I'm curious if rust has this problem. The problem I notice in npm land is many developers have no taste. Example, there's a library for globbing call glob. You'd think it would just be a function that does globbing but no, the author decided it should ALSO be a standalone commandline executable and so includes a large commandline option parser. They could have easily made a separate commandline tool that include a library that does the glob but no, this is a common and shit pattern in npm. I'd say easily 25% or more of all "your dependencies are out of date" messages are related to the argument parcing for the commandline tool in these libraries. That's just one example. Also there's arguably design. Should a 'glob' library actually read the file system and give you filenames or should it just tell you if a string matches a glob and leave the reset to you? I think it's better design to do the later, the simplest thing. This means less dependencies and more flexibility. I don't have to hack it or add option to use my own file system (like for testing). I can use it with a change monitoring system, etc... And, I'm sure there are tons of devs that like the glob is a "Do everything for me" library instead of a "do one specific thing" library which makes it worse because you get more "internet points" the more your library doesn't require the person using it to be a good dev. I can't imagine it's any different in rust land, except maybe for the executable thing. There's just too many devs and all of them, including myself, don't always make the best choices.

kazinator 1y ago
> Should a 'glob' library actually read the file system and give you filenames The POSIX glob function after which these things are named traverses the filesystem and matches directory entries. The pure matching function which matches a glob pattern against a filename-like string is fnmatch. But yes, the equivalent of fnmatch should be a separate module and that could be a dependency of glob. Nobody should be trying to implement glob from scratch using a fnmatch-like function and directory traversal. It is not so trivial. glob performs a traversal that is guided by the pattern. It has to break the pattern into path components. It knows that "*/*/*" has three components and so the traversal will only go three levels deep. Also "dir/*" has a component which is a fixed match, and so it just has to open "dir" without scanning the current directory; if that fails, glob has failed. If the double star ** is supported which matches multiple components, that's also best if it likewise integrated into glob. If brace expansion is supported, that adds another difficulty because different branches of a brace can have different numbers of components, like {*/x,*/*/x,*/*/*/x}. To implement glob, it would greatly help us to have brace expansion as a separate function which expands the braces, producing multiple glob patterns, which we can then break into path components and traverse.
- hinkley 1y ago
  They eventually fixed it but grunt once upon a time used a glob implementation that could not short circuit on wildcards in ignore patterns. So I caught it scanning the node-modules directory and then dropping every file it found because it matched on “node_modules/**”. Builds got a lot faster when I pushed that update out. There’s a lot of stupid ways to implement glob and only a couple of smart ones.
- Olreich 1y ago
  > But yes, the equivalent of fnmatch should be a separate module and that could be a dependency of glob. Interesting, lets look at fnmatch: https://pubs.opengroup.org/onlinepubs/9699919799/functions/fnmatch.html https://pubs.opengroup.org/onlinepubs/9699919799/functions/f... Well, fnmatch really does two things, it parses the pattern and then applies that to a string, so really, there should be a "ptnparse" library that handles the pattern matching that fnmatch has a dependency. Though, thinking it through, the "ptnparse" library is responsible for patterns matching single characters and multiple characters. We should split that up into "singleptn" and "multiptn" libraries that ptnparse can take as dependencies. Oh, and those flags that fnmatch takes makes fnmatch work in several different ways, let's decompose those into three libraries so that we only have to pull in the matcher we care about: pthmatch, nscmatch, and prdmatch. Then we can compose those libraries based on what we want in fnmatch. This is perfect, now if we don't care about part of the fnmatch functionality, we don't have to include it! /s This decomposition is how we wind up with the notorious leftpad situation. Knowing when to stop decomposing is important. fnmatch is a single function that does less than most syscalls. We can probably bundle that with a few more string functions without actually costing us a ton. Glob matching at a string level probably belongs with all the other string manipulation functions in the average "strings" library. Importantly, my suggestion that fnmatch belongs in a "strings" library does align with your suggestion that fnmatch shouldn't be locked into a "glob" library that also includes the filesystem traversal components.
tough 1y ago
Bun has a glob too https://bun.sh/docs/api/glob https://bun.sh/docs/api/glob
hedora 1y ago
Historically, the borrow checker has been a good shield against developers that have no taste. Not sure how long that’ll last.
- lionkor 1y ago
  Not really, there are plenty of large libraries today that were designed by complete boneheads. Granted, you only notice if you know that domain very well.
- bsaul 1y ago
  macro seems to compensate for that. there's definitely a "C++ template programming" vibe in some libraries.
- devnullbrain 1y ago
  Dynamically typed languages do the opposite.
IshKebab 1y ago
Yeah that's one huge advantage Rust has over NPM - Rust developers are a lot more skilled and crates are generally much higher quality.
- bsaul 1y ago
  Random remark: I've noticed the quality of rust libraries too. Which made me really surprised to see the overengineered mess that is the async-openai crate. how can one take an api as simple as openai's one, and turn it to this steaming pile of manure ? in the end, i used reqwest and created my queries manually. I guess that's what everyone does...
  - wizzwizz4 1y ago
    The kinds of people who think OpenAI's tech is worth touching with a bargepole are generally not the kinds of people who develop and maintain high-quality Rust libraries.
    - bsaul 1y ago
      i get it, but openai being the hotest stuff happening in software for the past.. x years , i would have assumed there was some kind of official client correctly maintained for rust. I was a bit shocked to be honest. edit : i originally misread your comment. OpenAI is an important tech, no matter what you think of the company itself. Being able to easily interface with their api is important.
- hombre_fatal 1y ago
  Maybe that were true back when Rust wasn't mainstream on social media nor across tech influencer videos, but it's not true anymore. https://crates.io/search?q=is-even https://crates.io/search?q=is-even
  - detaro 1y ago
    Oh no, people in the Rust community make jokes, how unprofessional!!111 You'll notice these packages are not actually used by anything.
jll29 1y ago
> I can't imagine it's any different in [R]ust land Taste is important; programmers with good architectural taste tend to use languages that support them in their endeavour (like Rust or Zig) or at least get out of the way (C). So I would argue the problems you list are statistically less often the case than in certain other languages (from COBOL to JavaScript). > There's just too many devs and all of them, including myself, don't always make the best choices. This point you raise is important: I think an uncoordinated crowd of developers will create a "pile of crates" ("bazaar" approach, in Eric Raymond's terminology), and a single language designer with experience will create a more uniform class library ("cathedral" approach). Personally, I wish Rust had more of a "batteries included" standard library with systematically named and namespaced official crates (e.g. including all major data structures) - why not "stdlib::data_structures::automata::weighted_finite_state_transducer" instead of a confusing set of choices named "rustfst-ffi", "wfst", ... ? Ideally, such a standard library should come with the language at release. But the good news is it could still be devised later, because the Rust language designers were smart enough to build versioning with full backwards compatibility (but not technical debt) into the language itself. My wish for Rust 2030 would be such a stdlib (it could even be implemented using the bazaar of present-day crates, as long as that is hidden from us).
mexicocitinluez 1y ago
> The problem I notice in npm land is many developers have no taste. Programming is not the same as hanging out some hoity-toity art gallery. If someone critiqued my software dev by saying I had "no taste", I'd cringe so hard I'd turn into a black hole. I know this is hackernews, but this reeks of self-importance.
- FredPret 1y ago
  Engineering is a form of art where the engineer makes many decisions, large and small, where optimality cannot be proven. Taste most certainly plays a role, and there are engineering products that clearly show good or poor taste. Unfortunately this particular art form requires fluency in mathematics and the sciences/computers, so it’s very inaccessible.
  - mexicocitinluez 1y ago
    > Taste most certainly plays a role, and there are engineering products that clearly show good or poor taste. No. Get over yourself.
    - FredPret 1y ago
      Wow, I’m convinced! In all seriousness: why do you insist on an intellectual totem pole with art being more “hoity toity” as you say, and more worthy? And if so, why is engineering below painting? In mathematics and some hard sciences, and in absolutely elementary aspects of other fields like programming, things can be proven to be right or wrong. In every other endeavour, we need to repeatedly apply fuzzy judgements. Some do this poorly, some do it well. Just in case you actually are interested in nuance, here’s some more on the topic: https://www.paulgraham.com/hp.html https://www.paulgraham.com/hp.html https://nealstephenson.substack.com/p/idea-having-is-not-art https://nealstephenson.substack.com/p/idea-having-is-not-art
      - mexicocitinluez 1y ago
        > why do you insist on an intellectual totem pole with art being more “hoity toity” as you say, and more worthy More worthy? Where are you getting that? Am i responding to a bot?
kibwen 1y ago
We don't need to speak in hypotheticals, we can just look at the glob crate: https://crates.io/crates/glob https://crates.io/crates/glob 213M downloads, depends on zero external crates, one source file (a third of which is devoted to unit tests), and developed by the rust-lang organization itself (along with a lot of crates, which is something that people tend to miss in this discussion).
- hu3 1y ago
  glob was just an example. They weren't asking about a specific crate. Also this crate is from official rust lang repo, so much less prone to individualistic misbehaving. A bad example all around.
  - deleted 1y ago
    [deleted]
  - timeinput 1y ago
    I think the parent was suggesting comparing and contrasting the glob dependency in rust, and npm. The one off isn't useful, but picking ten random, but heavily used packages probably is. The parent didn't really mention what the node version looked like though. The npm glob package has 6 dependencies (those dependencies have 3+ dependencies, those sub dependencies have 6+ dependencies, ...) As you point out the rust crate is from the official repo, so while it's not part of the standard library, it is maintained by the language maintenance organization. Maybe that could make it a bad example, but the npm one is maintained by the inventor of npm, and describes him self as "I wrote npm and a pretty considerable portion of other node related JavaScript that you might use.", so I would say that makes it a great example because the people who I would expect care the most about the language are the package maintainers of these packages, and are (hopefully) implementing what they think are the best practices for the languages, and the eco-systems.
  - kibwen 1y ago
    > Also this crate is from official rust lang repo, so much less prone to individualistic misbehaving. To reiterate, lots of things that people in this thread are asking the language to provide are in fact provided by the rust-lang organization: regex, serde, etc. The goalposts are retreating over the horizon. Rust's primary sin here is that it makes dependency usage transparent to the end-user. Nobody wants to think about how many libraries they depend upon and how many faceless people it takes to maintain those libraries, so they're uncomfortable when Rust shows you. This isn't a Rust problem, it's a software complexity problem.
blub 1y ago
What you’re describing regarding glob is not lack of taste, it’s an architectural “bug”. Taste is what Steve Jobs was referring to when he said Microsoft had none. In software it’s defined by a humane, pleasant design that almost(?) anybody can appreciate. Programming languages cannot be tasteful, because they require time and effort to learn and understand. Python has some degree elegance and Golang’s simplicity has a certain je ne sais quoi… but they’re not really fitting the definition. Still, some technologies such as git, Linux or Rust stand out as particularly obscure even for the average developer, not just average human.
throwitaway1123 1y ago
It's worth pointing out that Node has a built in globbing function: https://nodejs.org/docs/latest-v24.x/api/fs.html#fspromisesglobpattern-options:~:text=dir1/%20dir2/.-,fsPromises.glob(pattern%5B%2C%20options%5D),-%23 https://nodejs.org/docs/latest-v24.x/api/fs.html#fspromisesg... > Also there's arguably design. Should a 'glob' library actually read the file system and give you filenames or should it just tell you if a string matches a glob and leave the reset to you? There's a function in Node's stdlib that does this as well (albeit it's marked as experimental): https://nodejs.org/docs/latest-v24.x/api/path.html#pathmatchesglobpath-pattern:~:text=COPY-,path.matchesGlob(path%2C%20pattern),-%23 https://nodejs.org/docs/latest-v24.x/api/path.html#pathmatch...

kazinator 1y ago

> Should a 'glob' library actually read the file system and give you filenames The POSIX glob function after which these things are named traverses the filesystem and matches directory entries. The pure matching function which matches a glob pattern against a filename-like string is fnmatch. But yes, the equivalent of fnmatch should be a separate module and that could be a dependency of glob. Nobody should be trying to implement glob from scratch using a fnmatch-like function and directory traversal. It is not so trivial. glob performs a traversal that is guided by the pattern. It has to break the pattern into path components. It knows that "*/*/*" has three components and so the traversal will only go three levels deep. Also "dir/*" has a component which is a fixed match, and so it just has to open "dir" without scanning the current directory; if that fails, glob has failed. If the double star ** is supported which matches multiple components, that's also best if it likewise integrated into glob. If brace expansion is supported, that adds another difficulty because different branches of a brace can have different numbers of components, like {*/x,*/*/x,*/*/*/x}. To implement glob, it would greatly help us to have brace expansion as a separate function which expands the braces, producing multiple glob patterns, which we can then break into path components and traverse.

hinkley 1y ago
They eventually fixed it but grunt once upon a time used a glob implementation that could not short circuit on wildcards in ignore patterns. So I caught it scanning the node-modules directory and then dropping every file it found because it matched on “node_modules/**”. Builds got a lot faster when I pushed that update out. There’s a lot of stupid ways to implement glob and only a couple of smart ones.
Olreich 1y ago
> But yes, the equivalent of fnmatch should be a separate module and that could be a dependency of glob. Interesting, lets look at fnmatch: https://pubs.opengroup.org/onlinepubs/9699919799/functions/fnmatch.html https://pubs.opengroup.org/onlinepubs/9699919799/functions/f... Well, fnmatch really does two things, it parses the pattern and then applies that to a string, so really, there should be a "ptnparse" library that handles the pattern matching that fnmatch has a dependency. Though, thinking it through, the "ptnparse" library is responsible for patterns matching single characters and multiple characters. We should split that up into "singleptn" and "multiptn" libraries that ptnparse can take as dependencies. Oh, and those flags that fnmatch takes makes fnmatch work in several different ways, let's decompose those into three libraries so that we only have to pull in the matcher we care about: pthmatch, nscmatch, and prdmatch. Then we can compose those libraries based on what we want in fnmatch. This is perfect, now if we don't care about part of the fnmatch functionality, we don't have to include it! /s This decomposition is how we wind up with the notorious leftpad situation. Knowing when to stop decomposing is important. fnmatch is a single function that does less than most syscalls. We can probably bundle that with a few more string functions without actually costing us a ton. Glob matching at a string level probably belongs with all the other string manipulation functions in the average "strings" library. Importantly, my suggestion that fnmatch belongs in a "strings" library does align with your suggestion that fnmatch shouldn't be locked into a "glob" library that also includes the filesystem traversal components.

tough 1y ago

Bun has a glob too https://bun.sh/docs/api/glob https://bun.sh/docs/api/glob

hedora 1y ago

Historically, the borrow checker has been a good shield against developers that have no taste. Not sure how long that’ll last.

lionkor 1y ago
Not really, there are plenty of large libraries today that were designed by complete boneheads. Granted, you only notice if you know that domain very well.
bsaul 1y ago
macro seems to compensate for that. there's definitely a "C++ template programming" vibe in some libraries.
devnullbrain 1y ago
Dynamically typed languages do the opposite.

IshKebab 1y ago

Yeah that's one huge advantage Rust has over NPM - Rust developers are a lot more skilled and crates are generally much higher quality.

bsaul 1y ago
Random remark: I've noticed the quality of rust libraries too. Which made me really surprised to see the overengineered mess that is the async-openai crate. how can one take an api as simple as openai's one, and turn it to this steaming pile of manure ? in the end, i used reqwest and created my queries manually. I guess that's what everyone does...
- wizzwizz4 1y ago
  The kinds of people who think OpenAI's tech is worth touching with a bargepole are generally not the kinds of people who develop and maintain high-quality Rust libraries.
  - bsaul 1y ago
    i get it, but openai being the hotest stuff happening in software for the past.. x years , i would have assumed there was some kind of official client correctly maintained for rust. I was a bit shocked to be honest. edit : i originally misread your comment. OpenAI is an important tech, no matter what you think of the company itself. Being able to easily interface with their api is important.
hombre_fatal 1y ago
Maybe that were true back when Rust wasn't mainstream on social media nor across tech influencer videos, but it's not true anymore. https://crates.io/search?q=is-even https://crates.io/search?q=is-even
- detaro 1y ago
  Oh no, people in the Rust community make jokes, how unprofessional!!111 You'll notice these packages are not actually used by anything.

jll29 1y ago

> I can't imagine it's any different in [R]ust land Taste is important; programmers with good architectural taste tend to use languages that support them in their endeavour (like Rust or Zig) or at least get out of the way (C). So I would argue the problems you list are statistically less often the case than in certain other languages (from COBOL to JavaScript). > There's just too many devs and all of them, including myself, don't always make the best choices. This point you raise is important: I think an uncoordinated crowd of developers will create a "pile of crates" ("bazaar" approach, in Eric Raymond's terminology), and a single language designer with experience will create a more uniform class library ("cathedral" approach). Personally, I wish Rust had more of a "batteries included" standard library with systematically named and namespaced official crates (e.g. including all major data structures) - why not "stdlib::data_structures::automata::weighted_finite_state_transducer" instead of a confusing set of choices named "rustfst-ffi", "wfst", ... ? Ideally, such a standard library should come with the language at release. But the good news is it could still be devised later, because the Rust language designers were smart enough to build versioning with full backwards compatibility (but not technical debt) into the language itself. My wish for Rust 2030 would be such a stdlib (it could even be implemented using the bazaar of present-day crates, as long as that is hidden from us).

mexicocitinluez 1y ago

> The problem I notice in npm land is many developers have no taste. Programming is not the same as hanging out some hoity-toity art gallery. If someone critiqued my software dev by saying I had "no taste", I'd cringe so hard I'd turn into a black hole. I know this is hackernews, but this reeks of self-importance.

FredPret 1y ago
Engineering is a form of art where the engineer makes many decisions, large and small, where optimality cannot be proven. Taste most certainly plays a role, and there are engineering products that clearly show good or poor taste. Unfortunately this particular art form requires fluency in mathematics and the sciences/computers, so it’s very inaccessible.
- mexicocitinluez 1y ago
  > Taste most certainly plays a role, and there are engineering products that clearly show good or poor taste. No. Get over yourself.
  - FredPret 1y ago
    Wow, I’m convinced! In all seriousness: why do you insist on an intellectual totem pole with art being more “hoity toity” as you say, and more worthy? And if so, why is engineering below painting? In mathematics and some hard sciences, and in absolutely elementary aspects of other fields like programming, things can be proven to be right or wrong. In every other endeavour, we need to repeatedly apply fuzzy judgements. Some do this poorly, some do it well. Just in case you actually are interested in nuance, here’s some more on the topic: https://www.paulgraham.com/hp.html https://www.paulgraham.com/hp.html https://nealstephenson.substack.com/p/idea-having-is-not-art https://nealstephenson.substack.com/p/idea-having-is-not-art
    - mexicocitinluez 1y ago
      > why do you insist on an intellectual totem pole with art being more “hoity toity” as you say, and more worthy More worthy? Where are you getting that? Am i responding to a bot?

kibwen 1y ago

We don't need to speak in hypotheticals, we can just look at the glob crate: https://crates.io/crates/glob https://crates.io/crates/glob 213M downloads, depends on zero external crates, one source file (a third of which is devoted to unit tests), and developed by the rust-lang organization itself (along with a lot of crates, which is something that people tend to miss in this discussion).

hu3 1y ago
glob was just an example. They weren't asking about a specific crate. Also this crate is from official rust lang repo, so much less prone to individualistic misbehaving. A bad example all around.
- deleted 1y ago
  [deleted]
- timeinput 1y ago
  I think the parent was suggesting comparing and contrasting the glob dependency in rust, and npm. The one off isn't useful, but picking ten random, but heavily used packages probably is. The parent didn't really mention what the node version looked like though. The npm glob package has 6 dependencies (those dependencies have 3+ dependencies, those sub dependencies have 6+ dependencies, ...) As you point out the rust crate is from the official repo, so while it's not part of the standard library, it is maintained by the language maintenance organization. Maybe that could make it a bad example, but the npm one is maintained by the inventor of npm, and describes him self as "I wrote npm and a pretty considerable portion of other node related JavaScript that you might use.", so I would say that makes it a great example because the people who I would expect care the most about the language are the package maintainers of these packages, and are (hopefully) implementing what they think are the best practices for the languages, and the eco-systems.
- kibwen 1y ago
  > Also this crate is from official rust lang repo, so much less prone to individualistic misbehaving. To reiterate, lots of things that people in this thread are asking the language to provide are in fact provided by the rust-lang organization: regex, serde, etc. The goalposts are retreating over the horizon. Rust's primary sin here is that it makes dependency usage transparent to the end-user. Nobody wants to think about how many libraries they depend upon and how many faceless people it takes to maintain those libraries, so they're uncomfortable when Rust shows you. This isn't a Rust problem, it's a software complexity problem.

blub 1y ago

What you’re describing regarding glob is not lack of taste, it’s an architectural “bug”. Taste is what Steve Jobs was referring to when he said Microsoft had none. In software it’s defined by a humane, pleasant design that almost(?) anybody can appreciate. Programming languages cannot be tasteful, because they require time and effort to learn and understand. Python has some degree elegance and Golang’s simplicity has a certain je ne sais quoi… but they’re not really fitting the definition. Still, some technologies such as git, Linux or Rust stand out as particularly obscure even for the average developer, not just average human.

throwitaway1123 1y ago

It's worth pointing out that Node has a built in globbing function: https://nodejs.org/docs/latest-v24.x/api/fs.html#fspromisesglobpattern-options:~:text=dir1/%20dir2/.-,fsPromises.glob(pattern%5B%2C%20options%5D),-%23 https://nodejs.org/docs/latest-v24.x/api/fs.html#fspromisesg... > Also there's arguably design. Should a 'glob' library actually read the file system and give you filenames or should it just tell you if a string matches a glob and leave the reset to you? There's a function in Node's stdlib that does this as well (albeit it's marked as experimental): https://nodejs.org/docs/latest-v24.x/api/path.html#pathmatchesglobpath-pattern:~:text=COPY-,path.matchesGlob(path%2C%20pattern),-%23 https://nodejs.org/docs/latest-v24.x/api/path.html#pathmatch...