13 ms·
What was the point of [ “x$var” = “xval” ]? (2021)
- lolc 4y agoThis abomination reminds you to use a proper scripting language.
- junon 4y ago... or perhaps, as the article suggests, that parsing was hard when we first started doing it decades ago and not everyone got it right. If it wasn't abundantly clear, this is not an issue with any of the shells today.
- mschuster91 4y ago> If it wasn't abundantly clear, this is not an issue with any of the shells today. The least-common-denominator aka /bin/sh and its syntax will stick around for decades to come. You can't rely on any of the "modern" shells to be around, particularly if you write scripts that target a multitude of unix-like systems (macOS, xBSD, Linux).
- kmill 4y agoIt's amusing that parsing theory was already at least 1-2 decades old by the time they made this mistake (consider that ALGOL 60 had a complicated grammar, introducing BNF too). I'd chalk it up to this being a shell language that was evolved rather than designed.
- mananaysiempre 4y agoI wouldn’t say that parsing is hard (precedence parsing is especially not hard), but instead that logical operators and grouping inside test aka [ were both a bad idea (given a sufficiently complex expression, I expect you can still confuse a human as to what’s an operator and what’s a string even if POSIX is precise enough to dictate a single interpretation) and unnecessary (the shell itself has perfectly serviceable !, &&, ||, and grouping, though the ! is a bit awkward and seems to be a later addition). If you omit those, a fairly intuitive unambiguous parse is very easy: if we’re [, check last argument is ] and throw it away; if one argument, return whether it’s empty; otherwise, if two, the first is a unary operator, evaluate and return that; otherwise, if three, the second is a binary operator, evaluate and return that; otherwise fail. That’s it.
- hgsgm 4y agoThe OP gives two examples where it is not fixed, because bash is so poorly designed that the fixes are partial hacks.
- marcosdumay 4y ago> If it wasn't abundantly clear, this is not an issue with any of the shells today. I really doubt it. If the comparison parameters are undistinguishable from the operators, it's only a matter of enough creativity and people will to find new ways to break it.
- 1vuio0pswjnm7 4y agoLOL. And what how would one run that proper scripting language. On top of a shell? With a userland and kernel built with shell scripts? Or maybe Microsoft Windows? Perhaps with Windows one can avoid the Almquist shell. Otherwise, it's still there whether one chooses to use it or not. Scripts using this convention are common, at least on NetBSD.
- cxr 4y agoThis is such a bizarre response. A Python* script, to give one example, does not—at least in any meaningful sense—run on "top of a shell". The shell will accept the command string and fork the process, but that has nothing to do with the design of the shell language being discussed here, and the whole thing happens to be incidental, anyway—you could completely remove the Unix shell from the equation by just making the Python interpreter your shell if you wanted, which would sidestep every element of the already-tenuous line of reasoning that you're using here to try to make an argument... * which I abhor, by the way—just like GNU abhors man pages—but that doesn't make your argument any less absurd
- H4ZB7 4y agohe probably meant on the terminal but he's just LARPing like every other UN*X enthusiast. they like to create false dilemmas like "legacy" and "embedded"* to justify their bloated poorly implemented string flinging machine that has another 100KLOC to parse the input line (readline) imagine someone pitching shell programming today. the response would be: LOL so you mean each language will have a common subset of syntax that supposdely works the same across all of them? how can anyone ever program that way? * ironically, a big part of why embedded products are so flaky is just because of their shell scripts
- 1vuio0pswjnm7 4y agoThe only thing that makes UNIX appealing to me is the fact that I find all the alternatives are so much more annoying. If there is some UNIX alternative that is less bloated and slow, please share. NB. Almquist shell does not necessarily use readline. Neither NetBSD's sh nor Linux's dash use it. (Not to imply editline isn't bloated, too, but it's not nearly as bad.) Further, I did not mean "the terminal". I had to look up "LARP". FWIW, I do not play video games. I do not use readline. I do not use a "terminal emulator". I use textmode. These choices are not because I think highly of ash, editline, textmode, non-graphical computing or minimalism in general. They are because I find the alternatives (large scripting languages, readline, video games, terminal emulators) are really annoying. Why not share some of the programs H4ZB7 has written, so we can see how to do things the right way.
- papruapap 4y agoIt's funny how this community trash on JS at every chance they have but for some reason bash is off the hook. I am tired on seeing CI pipelines failing and Live environment crashing because typos in .sh scripts. I propose to switch to powershell every time I can but that still sounds as heresy for the devops guys.
- crabbone 4y agoLol... You are going to switch to PowerShell in initramfs? PowerShell is just another Perl / Python / Ruby. Except made by Microsoft, so it's got a ton of useless features, lacking some essential stuff and has bombastic syntax. It's not suited for the role of UNIX Shell. Not by a long shot. ---- On a larger note, the whole idea of needing a shell is bad. It comes from a defective (or rather overly simplistic and un-insightful) design of UNIX. A better system wouldn't need a distinction between a system programming language and a shell. You would just use one and the same thing for both purposes. If you are on Linux, Guile is supposed to be that, but... maybe it was supposed to be that?.. I'd still want it to take the role of Shell, even though it doesn't tick all my boxes.
- Kwpolska 4y agoNobody said anything about switching to PowerShell at the lowest levels. But even though it is verbose, and has some annoying quirks (like over-eager unpacking of one-item arrays), it's much nicer to write scripts in. You don't need sed/awk to parse output of ls and hope it's stable, just get whatever data you need from the object. What essential stuff is missing?
- bmacho 4y agoI am still waiting for a language that has variables, strings, loops and if-branches, and can pipe programs together. OIL and ABS are close but who uses those.
- ajross 4y agoThe weird thing is we were already there in the early 90's with perl, and as a community uniformly decided to walk away from it. Now we're "still waiting" for stuff people were running on SPARCstations?
- saltcured 4y agoI think all but the die hards got tired of the jokes. If only they had held on a little longer, those would have stopped when people forgot about modems and line-noise...
- ufo 4y agoCheck out Tcl! It's built around strings, with a command syntax similar to shell. The standard library "exec" command allows launching programs including pipes and redirects. Although it's a bit clunky (e.g. there's no way to quote a '|' or '>' argument) https://www.tcl-lang.org/man/tcl/TclCmd/exec.htm https://www.tcl-lang.org/man/tcl/TclCmd/exec.htm
- freedomben 4y agoIs tcl still mainstream? And what's the dependency install like on a standard system? I used tcl 15 years ago and it was pretty great, but the SDKs were already getting old and many falling unmaintained. I worried about the future of tcl
- ufo 4y agoOn Fedora, I think it might even come installed by default (possibly as a dependency of Python's Tkinter module). There's also Jim Tcl, an embeddable implementation with very few dependencies.
- Pxtl 4y agoEvery major admin scripting language I've ever used has so many foibles and edge-cases it makes me pine for full programming languages. What admin scripting languages bring to the table: 1) Easy access to executable binaries - no creating a Process object to describe it, just run it 2) Easy piping and filtering 3) A standard library designed for terse and easy-to-use access to administrative tasks. 4) Pleasant in REPL form. That's the good side. The bad side is that they're all such a collection of hideous hacks and edge-cases for basic concepts like variable scope and typing that even the simplest operations that would be hilariously trivial in a real language are utterly agonizing.
- JohnFen 4y agoThere are many situations where you don't get to choose what scripting language you're going to use.
- citrin_ru 4y agoA posix shell script written in 1990s would still work today and likely will work 10 years from now (given that external commands it uses had not breaking changes). Not many proper scripting languages have such track record - most of them either had many small backward incompatible changes or an update which breaks many things at once (like Python 2 -> 3). Though I have some 20+ years old Perl scripts which still work today but it is hard to find a language which attracts more haters than Perl.
- chrismorgan 4y agoDiscussed extensively last year: https://news.ycombinator.com/item?id=26776956 https://news.ycombinator.com/item?id=26776956 One key point there is that it’s still easy to trigger cases where you need this hack, because the cleverness is only stable for up to four arguments; so and, or, and parentheses all exceed the point of safety.
- topspin 4y agoSo keep doing it despite whomever it annoys. Also, throw some superfluous cats in to annoy them more.
- amluto 4y agoAll the cleverness and precedence rules in the article seem to ignore the fundamental problem: the test and [ binaries are regular programs, and they cannot distinguish between an argument that looks like “(“ and (. In both cases, they get the string “(“ in argv. So x serves as a form of quoting. If I were to potentially invoke a binary to evaluate an expression, I would quote any string-like variable (using the x hack) and I would do my best to avoid numeric comparisons. (Or use built-in shell features where the shell knows which values are quoted.)
- rascul 4y ago> All the cleverness and precedence rules in the article seem to ignore the fundamental problem: the test and [ binaries are regular programs, and they cannot distinguish between an argument that looks like “(“ and (. In both cases, they get the string “(“ in argv. Regular programs can make that distinction with no problem, the mistake is thinking that the outer quotes are part of the argument. In the context of the POSIX style shell, they're not, they signify that what is enclosed is one argument. So you'll have to use one of the various mechanisms the shell provides to indicate that you want to pass the quotes as an argument.
- deleted 4y ago[deleted]
- deleted 4y ago
- deleted 4y ago[deleted]
- tragomaskhalos 4y agoGuilty here, but have been doing so for so long that it's largely muscle memory at this point and the original rationale has been lost. In mitigation, it certainly used to be the case that you were likely to come across a broad variety of Unixy flavours and hence using highly conservative scripting constructs made sense; then after a while these become second nature, and unless they cause actual problems they just stay in the mental toolkit.
- mkovach 4y agoSame here! When I started off in the UNIX world, the shop had a collection of AIX, SunOS, Solaris, HP/UX and Digital UNIX. the x-hack was standard form and I spent a few years fixing errors with the x-hack (or converting the scripts to Perl). It is just second nature to use the x-hack in if's and case statements on this point. I have to try really hard to NOT use it. Of course, the folks with less gray in their hair and often confused by it, referring to it a "Olde Unix." I have approved plenty of PRs with the comment, "Updating mek's Olde Unix syntax."
- pixelbeat__ 4y agoOh I will reference this from https://www.pixelbeat.org/programming/shell_script_mistakes.html https://www.pixelbeat.org/programming/shell_script_mistakes.... where I also advise against this common archaic idiom
- pyentropy 4y agoWait... zsh which is quite a powerful shell that's now by default on macOS too, would say "(" is equal to ")" until 2016? Was it just single [ (test) or [[ as well?
- adql 4y agoWouldn't be surprised if it was just backward compatibility so bash/sh scripts work
- yrro 4y agoUsing [ is a mistake. Use [[ and relax!
- rurban 4y agoUnportable
- orra 4y agoPortability isn't binary. The question is, is it portable to the systems you want to support?
- digitalsushi 4y agoIt's portable until it pages out on an incident report. Then, we know it wasn't portable.
- sgbeal 4y ago> The question is, is it portable to the systems you want to support? "Want" isn't always the whole equation. Sysadmins are often called to support ancient OSes, in particular in environments which keep systems in operation long after their "best before" dates (e.g. internal IT in banking systems). There's literally no disadvantage, beyond aesthetics, to _not_ using the x-hack. For many of us older folks, using it is built into our muscle memory.
- rurban 4y agoWe need to support the most common OS in the world, debian. default shell: dash bashisms are forbidden in every corner of the world, sorry https://askubuntu.com/questions/1277922/what-are-syntax-differences-between-bash-and-dash https://askubuntu.com/questions/1277922/what-are-syntax-diff...
- orra 4y agoBy default, dash provides /bin/sh on Debian. But that doesn't prevent people targeting /bin/bash instead. I'll also note that BusyBox supports [[.
- rurban 4y agoYou'd never do [ "x$left" = "x$right" ], but [ x$left = x$right ]. The whole reason for the x-hack was to omit the double-quotes. With the dquotes you can of course leave the x out. [ "$left" = "$right" ]
- Karellen 4y agoRemember though, just because the shell on your rolling-release dev machine fixed this issue years ago, does not mean the shells on all your users/clients/legacy servers/hobby backports have fixed the issue too.
- MontyCarloHall 4y ago>However, the value was mostly gone by the mid-to-late 1990s, and the few remaining issues were cleaned up before 2010 — shockingly late, but still over a decade ago. Anyone running a shell that still requires the x-hack is deliberately maintaining a legacy system, at which point the onus is on them to get modern tooling working on their machine.
- pfortuny 4y agoBeing forgiving towards the user is one of the important principles of programming. Because the user may not be able to “upgrade” his system. Windows 95 is still very much alive. Embedded-like systems are a thing.
- Macha 4y agoYet asking for windows 95 support on most open source bug trackers will get a reaction between being laughed off the tracker or "do it yourself, but we'll reject the PR if it makes too much of a mess". I don't see why legacy commercial Unix deserves any higher level of support these days.
- pfortuny 4y agoDid not say so. Only that if one does do it, one is being understanding. Did not say it has to be done.
- JohnFen 4y agoJust because some projects (OSS or not) decide on a certain policy doesn't mean that the policy is necessarily good, or that any other projects should follow suit. Whether or not this is a reasonable stance depends on the project and the target demographic.
- mauvehaus 4y agoIf you aren't quoting the left hand side, you still need it if the left side is an empty string: $ foo="" $ [ $foo = "" ] -sh: [: =: unary operator expected $ [ x$foo = "x" ] $ echo $? 0 I'm not arguing that this is a good idea, merely observing that this case remains. Empty strings had always been the case that I'd been taught required the x-hack.
- misnome 4y agoIsn't this one of the things that bash [[ ]] explicitly fixes?
- hgsgm 4y agoI hope not, because $empty is necessary when you actually want to elide a missing value, like Python's `filter`
- rullelito 4y agoWhat a horrible language:x
- digitalsushi 4y agoIt's horrible but it's home for a lot of us.
- Already__Taken 4y agomore like holm for you.
- Spivak 4y agoThe parent I think is getting two things mixed up. [[ $var == "" ]] works as you describe and as expected. What the parent is talking about is when you have an invocation like my_program $opt my_program "$opt" Which when empty will evaluate to my_program my_program "" respectively.
- hgsgm 4y ago> You can still see this on macOS bash today: $ str="-e" $ [ \( ! "$str" \) ] bash: [: `)' expected, found ] # bash > POSIX fixes all these ambiguities for up to 4 parameters So... It's not fixed, and the hack is still needed?
- zimpenfish 4y agoTo be fair, that's bash 3.2.57 which was released 2014-11-07. Oldest bash after that I have available is cygwin bash 4.4.12 (2017-10-13) which doesn't have the issue. Just install a newer bash using ports or homebrew and bob is your shell uncle.
- aflag 4y agoIf your script is going to be used by people on macs it’s probably easier to use the hack than ask them to update. Given all she’ll weirdness, I don’t even think this is the most offensive
- mannyv 4y agoLinux's not unix!
- msla 4y agoLegally, FreeBSD isn't Unix. There is, however, Inspur K-UX, a Linux distro which is a Unix.
- jmmv 4y agoHere is something I wrote a while ago which may help provide some context and that explains the differences between test, [ and [[ : https://jmmv.dev/2020/03/test-bracket.html https://jmmv.dev/2020/03/test-bracket.html
- russfink 4y agoHelpful, thanks!
- cabalamat 4y agoIt is for this sort of reason that I use Python instead of shell scripts.
- scbrg 4y agoYes. No gotchas there. >>> foo = b'foo' >>> foo[0] == b'f' False Don't get me wrong. I love Python, it has been my daily driver for 20+ years (and it pays my salary), but it does have its own set of cognitive burdens. Also, shell scripts I wrote in the 90:s work today. Python scripts - not so much.
- deleted 4y ago[deleted]
- deleted 4y ago[deleted]
- bagels 4y agoBash is the hardest language to write comparisons in if you're not doing it every day. The syntax is so unlike anything else.
- freedomben 4y ago> Bash is the hardest language to write comparisons in if you're not doing it every day. The syntax is so unlike anything else. worse than brainfuck? I find that hard to believe... on a serious note though, I don't agree. the syntax is not that different from the awk -> perl -> ruby heritage. Just a few constraints/conventions due to whitespace significance that other langs don't have.
- JohnFen 4y agoI agree. There are genuinely weird things about bash that increases the difficulty of writing for it, but I never really counted comparisons as one of them.
- klyrs 4y ago> worse than brainfuck? I find that hard to believe... Malbolge perhaps. Brainfuck is tedious, but it's a language of 8 symbols; there's nothing to forget. But doing comparisons in bash calls for a whole quickref of its own.
- pengaru 4y agoI only ever encounter this pattern in autotools-related shell code, stuff that ultimately ends up in configure scripts. There it arguably makes sense to be so robust against old/broken shells, since the whole point is to successfully configure the build on a broad spectrum of environments.
- kazinator 4y agoHere is a question: what is the point of if test x$var = val ; ... instead of using the [ command? Some years ago, I checked: the [ alias for test was already in the Bourne shell in 1979. Assume the year is 1999. What real portability problem do you solve by avoiding [ in favor of test? Real meaning that the rest of the script, and the software stack the script is related to, runs on a system where [ wouldn't work.
- dundarious 4y agoI always use test: - it looks like every other thing in sh (`[` is the outlier) - it prevents confusion like putting `&&` inside the `[ ... ]` brackets (understandable that `[` works that way if you think about it, but it’s another way it’s an outlier) - mildly discourages use of one particular bash-ism, `[[`, as it works slightly differently, and I prefer POSIX sh unless there is a good justification - it doesn’t require a closing `]` I don’t use it for what I think you mean by portability, that is, because `test` exists whereas `[` does not, or anything like that.
- ezekiel68 4y agoFor all of the arcanities, things like this really make me admire the pragmatic attitude of the grey beards. Kind of like how the limited register count in early CPUs led to "of course" reusing one of your operand registers to store an instruction result.