5 ms·
The history and use of /etc/glob in early Unixes
- JimmyWilliams1 2y ago[flagged]
- rednafi 2y agoThis reads like slop for some reasons; even to my non-native brain.
- keybored 2y agohttps://news.ycombinator.com/item?id=42575204 https://news.ycombinator.com/item?id=42575204
- marcus0x62 2y agoI wonder what the point of these accounts is - they show up in almost every post now. If the goal is farming karma, they aren't doing a very good job.
- yjftsjthsd-h 2y ago> PS: I don't know why expanding shell wildcards used a separate program in V6 and earlier, but part of it may have been to keep the shell smaller and more minimal so that it required less memory. See, I thought it was a nice separation of concerns and wondered why we lost such a nice approach, until I read: > How escaping wildcards works in the V5 and V6 shell is that all characters in commands and arguments are restricted to being seven-bit ASCII. The shell and /etc/glob both use the 8th bit to mark quoted characters, which means that such quoted characters don't match their unquoted versions and won't be seen as wildcards by either the shell at which point I suddenly became a fan of ditching it. I do wonder if there's not some better way to factor that functionality out...
- deleted 2y ago[deleted]
- hnlmorg 2y agoThe way Murex works is each parameter is first compiled into an AST, and then globing only works against the unquoted tokens. Globbing is also a separate built in, which allows for other types of wildcard matches like regex too. Eg https://murex.rocks/tour.html#filesystem-wildcards-globbing https://murex.rocks/tour.html#filesystem-wildcards-globbing So you have have the best of both worlds: inline globbing for convenience and also wildcard matching as a function too.
- eru 2y ago> at which point I suddenly became a fan of ditching it. I do wonder if there's not some better way to factor that functionality out... Just use backslash escaping like we do practically everywhere else in the Unix world?
- rini17 2y agoThat's kind of cure worse than disease. Just ditch escaping completely.
- yjftsjthsd-h 2y agoIf you completely ditch escaping, how do you handle filenames that contain special characters (in this context, mostly ? and *, but ()[] are also perennial favorites)? And to preempt the most obvious answer: No, you can't just ban them, because existing OSs and filesystems allow them and you need interoperability.
- rini17 2y agoThere are ways, no idea why doing anything here is so reviled. Find and xargs can delimite filenames by NUL, which is not allowed in filenames. Best practice in SQL was to abandon parameters escaping completely and pass them out of band. For internal representation, use array datastructures with length information. Actually, would it be that bad, to ban * and ? in filenames? If you accept them in the name of interop, something inevitably breaks later. Better to fail upfront. Many applications do sanitize filenames already and when they need to use binary data as file name, convert it to hex instead. It's a hassle otherwise.
- timewizard 2y agoSweet. I use xterm.js a lot and have a "shell backbone" that I use to make shell based access to APIs, S3 and other things "cloud." This is essentially how I implement globbing as well. The convenience is that you can run glob by itself to get an idea of exactly what kind of automated nightmare you are about to kick off. Anyways.. mine currently has V3 behavior. My shell command exec routine could actually benefit from that hack. What's old is new again?
- ginko 2y agoWhy is there a period after etc in the title? Another example of HN's stupid automated title editing?
- mkl 2y agoProbably the submitter typed it on a phone instead of copy-paste and "etc" got autoincorrected.
- rollcat 2y agoThis is php.ini level of madness, and I'm glad it's gone from (semi-)modern shells. A formal (e.g. programming) language should be defined in its entirety by its formal grammar, its semantics by a formal spec, etc. There's barely any good reason to let the system administrator change the logic and semantics of deployed code. You could argue that Lisp reader macros also somewhat violate this rule. As a longtime Lisp fan, I dislike reader macros, but I'm more conflicted about macros in general. A good macro system should aim to provide enough context for IDEs and LSPs to aid the developer, but Lisp macros are entirely about just transforming the AST. It's usually just better to evolve the language.
- JdeBP 2y agoIt's not there to give the system administrator flexibility. It's there because early Unix was heavily constrained, and doing thing with lots of little overlays (and what was decades later known as "Bernstein chaining") rather than 1 big program was the way to architect stuff. exit(1), goto(1), and if(1) were all external commands in the Thompson shell. * https://v6sh.org https://v6sh.org
- rollcat 2y agoI would argue with almost anyone else, that this is a poor design, but... Thank you for your perspective, work, and contributions.
- hnlmorg 2y agoThe other thing to bear in mind is that it’s undergone literally decades of evolution while still being backwards compatible. The shells weren’t originally intended to be Turing complete. They were just a job launcher. What you use today would have been unimaginable when these shells were first designed. Whereas all other programming languages have had a drastically smaller evolution in comparison and yet still had a worse compatibility story. It’s very easy to be critical of the Bourne shell (and compatible shells too) because they are archaic by modern standards. But they weren’t written to solve modern problems. So it’s like looking at a bicycle and complaining how the designers didn’t design a sports car while ignoring the fact that technology didn’t exist and still push bikes are good enough for millions to use daily.
- gjvc 2y agobinaries in /etc/ -- i mean __really__
- NekkoDroid 2y agoFun fact: the linux kernel itself actually also looks for `/etc/init` before it even looks for `/bin/init` https://github.com/torvalds/linux/blob/4a5df37964673effcd9f84041f7423206a5ae5f2/init/main.c#L1525-L1529 https://github.com/torvalds/linux/blob/4a5df37964673effcd9f8...
- tedunangst 2y agoYes, really. That's what /etc was for.
- gjvc 2y agoI know. I'm saying it's sick. I hate computers.
- kps 2y agoWhy sick? That was the directory for binaries that weren't meant to be run directly — `getty`, `login`, etc. Today there's much more software, so some things got moved into finer-grained locations like /libexec and /sbin. That wasn't the case in the /etc/glob era when the entire UNIX system was smaller than today's average web page.
- gjvc 2y agoand /sbin was full
- stevekemp 2y agoEven now you'll come across this, for example "/etc/rmt" probably exists, and other tape-related binaries if installed.
- imglorp 2y agoUser defined functions were implemented similarly as external execs in early shells. As the script was parsed, functions were dropped into /tmp without their wrappings and then called as external programs. Since they would still reference parameters as $1, $2 etc, it just worked: function bodies and standalone sh scripts had the same interface! Such a clever idea to avoid managing an interpreted call stack in the parent.
- deleted 2y ago[deleted]
- amelius 2y agoRecent versions of Bash don't expand the * (et cetera) patterns when there is no match, which although sometimes useful, I still feel it's a hack.
- Joker_vD 2y agoThat's been around since the original Bourne shell; /etc/glob, from what I can see from its source, would refuse to run the command if the resulting expansion turned out completely empty; and the globs with no matches would be simply removed.
- amelius 2y agoThat's not how it works in recent Ubuntu releases. If there is no match, the command runs with the wildcard chars not substituted. # echo foo*bar foo*bar
- Joker_vD 2y agoYes, this current behaviour was introduced by the original Bourne shell and then it stuck for some silly reason or another (it probably has some fringe use cases but they elude me). Thompson's original shell, or rather, /etc/glob, at various versions implemented the mix of behaviours that would later be reintroduced as nullglob and failglob options in Bash.
- pwg 2y agoThe action to take upon no match is configurable in recent Bash versions. The 'failglob' shopt option will cause an error to be generated if a glob matches nothing. The 'nullglob' shopt option toggles between no match expanding to an empty string and the traditional default of no match leaving the glob characters untouched.
- miohtama 2y agoThe linked C source file is an excellent example of ancient C, when it was still more closer to high level assembly: https://www.tuhs.org/cgi-bin/utree.pl?file=V2/cmd/glob.c https://www.tuhs.org/cgi-bin/utree.pl?file=V2/cmd/glob.c
- tomtomtom777 2y agoAnd when buffer overflows were (attempted to be) avoided by guestimating a large enough buffer size.
- LeFantome 2y agoI assume that you are referring to the liberal use of “goto”. Of course, “if”, “while”, and even “switch” are also used. Quite the mix. Directly calling into system calls (“write”) is interesting.
- kps 2y agoThe modern form of stdio only appear in Seventh Edition Unix.
- quuxplusone 2y agowrite(2) is POSIX. That's not "directly calling into a system call"; it's a normal C API from the POSIX header <unistd.h>.