2 ms·

> Should a 'glob' library actually read the file system and give you filenames The POSIX glob function after which these things are named traverses the filesys

> Should a 'glob' library actually read the file system and give you filenames The POSIX glob function after which these things are named traverses the filesystem and matches directory entries. The pure matching function which matches a glob pattern against a filename-like string is fnmatch. But yes, the equivalent of fnmatch should be a separate module and that could be a dependency of glob. Nobody should be trying to implement glob from scratch using a fnmatch-like function and directory traversal. It is not so trivial. glob performs a traversal that is guided by the pattern. It has to break the pattern into path components. It knows that "*/*/*" has three components and so the traversal will only go three levels deep. Also "dir/*" has a component which is a fixed match, and so it just has to open "dir" without scanning the current directory; if that fails, glob has failed. If the double star ** is supported which matches multiple components, that's also best if it likewise integrated into glob. If brace expansion is supported, that adds another difficulty because different branches of a brace can have different numbers of components, like {*/x,*/*/x,*/*/*/x}. To implement glob, it would greatly help us to have brace expansion as a separate function which expands the braces, producing multiple glob patterns, which we can then break into path components and traverse.

hinkley 1y ago
They eventually fixed it but grunt once upon a time used a glob implementation that could not short circuit on wildcards in ignore patterns. So I caught it scanning the node-modules directory and then dropping every file it found because it matched on “node_modules/**”. Builds got a lot faster when I pushed that update out. There’s a lot of stupid ways to implement glob and only a couple of smart ones.
Olreich 1y ago
> But yes, the equivalent of fnmatch should be a separate module and that could be a dependency of glob. Interesting, lets look at fnmatch: https://pubs.opengroup.org/onlinepubs/9699919799/functions/fnmatch.html https://pubs.opengroup.org/onlinepubs/9699919799/functions/f... Well, fnmatch really does two things, it parses the pattern and then applies that to a string, so really, there should be a "ptnparse" library that handles the pattern matching that fnmatch has a dependency. Though, thinking it through, the "ptnparse" library is responsible for patterns matching single characters and multiple characters. We should split that up into "singleptn" and "multiptn" libraries that ptnparse can take as dependencies. Oh, and those flags that fnmatch takes makes fnmatch work in several different ways, let's decompose those into three libraries so that we only have to pull in the matcher we care about: pthmatch, nscmatch, and prdmatch. Then we can compose those libraries based on what we want in fnmatch. This is perfect, now if we don't care about part of the fnmatch functionality, we don't have to include it! /s This decomposition is how we wind up with the notorious leftpad situation. Knowing when to stop decomposing is important. fnmatch is a single function that does less than most syscalls. We can probably bundle that with a few more string functions without actually costing us a ton. Glob matching at a string level probably belongs with all the other string manipulation functions in the average "strings" library. Importantly, my suggestion that fnmatch belongs in a "strings" library does align with your suggestion that fnmatch shouldn't be locked into a "glob" library that also includes the filesystem traversal components.

hinkley 1y ago

They eventually fixed it but grunt once upon a time used a glob implementation that could not short circuit on wildcards in ignore patterns. So I caught it scanning the node-modules directory and then dropping every file it found because it matched on “node_modules/**”. Builds got a lot faster when I pushed that update out. There’s a lot of stupid ways to implement glob and only a couple of smart ones.

Olreich 1y ago

> But yes, the equivalent of fnmatch should be a separate module and that could be a dependency of glob. Interesting, lets look at fnmatch: https://pubs.opengroup.org/onlinepubs/9699919799/functions/fnmatch.html https://pubs.opengroup.org/onlinepubs/9699919799/functions/f... Well, fnmatch really does two things, it parses the pattern and then applies that to a string, so really, there should be a "ptnparse" library that handles the pattern matching that fnmatch has a dependency. Though, thinking it through, the "ptnparse" library is responsible for patterns matching single characters and multiple characters. We should split that up into "singleptn" and "multiptn" libraries that ptnparse can take as dependencies. Oh, and those flags that fnmatch takes makes fnmatch work in several different ways, let's decompose those into three libraries so that we only have to pull in the matcher we care about: pthmatch, nscmatch, and prdmatch. Then we can compose those libraries based on what we want in fnmatch. This is perfect, now if we don't care about part of the fnmatch functionality, we don't have to include it! /s This decomposition is how we wind up with the notorious leftpad situation. Knowing when to stop decomposing is important. fnmatch is a single function that does less than most syscalls. We can probably bundle that with a few more string functions without actually costing us a ton. Glob matching at a string level probably belongs with all the other string manipulation functions in the average "strings" library. Importantly, my suggestion that fnmatch belongs in a "strings" library does align with your suggestion that fnmatch shouldn't be locked into a "glob" library that also includes the filesystem traversal components.