17 ms·

What was the point of [ “x$var” = “xval” ]?

414 points by JNRowe 6y ago

raverbashing 6y ago
> Amazingly, the x-hack could be used to work around certain bugs all the way up until 2015, seven years after StackOverflow wrote it off as an archaic relic of the past! Oof. If people think JS is bad wait until they try to do anything moderately complex in shell script
- iso1631 6y ago
  > If people think JS is bad wait until they try to do anything moderately complex in shell script The number of times I've started with a simple bash script and regretted it down the line. Now I refactor into perl far earlier in the process
- dale_glass 6y ago
  Yup. These days if it's more than a few lines, I write it in Perl instead if it's for personal use, or Python if it's going to be shared (Python just takes me more effort still). For all the vaunted Unix Philosophy it's amazing how clunky some of it is. Countless shell scripts still trip over spaces in filenames. Of those that don't, nearly all of them will still be confused by any unusual characters like newlines. If you want something done properly, you have to pull out a proper scripting language, to have readdir() and the ability to pass arguments directly to a process. Even Perl, which generally excels in such tasks has weird lapses in convenience. You can run a command with one line of code without the possibility of confusion with `system("ls", "-l", $dir)`, but you can't get its output that way. There's no version of `system` that'd allow you to both explicitly specify each argument to the process, and obtain its output. You either use backticks and risk quoting trouble, or need to use `open`, which is a lot more verbose. It's interesting that it took Microsoft to do a new approach in this regard. PowerShell has its amount of weirdness I really hate, such as that the environment escapes the program into the commandline, but it's really refreshing how it dispenses with brittle grep, awk and cut stuff.
  - mfontani 6y ago
    For Perl, you might want to look at https://metacpan.org/pod/IPC::Run3 https://metacpan.org/pod/IPC::Run3 You can: - specify each argument separately - get separate stdout and stderr - provide input
    - dale_glass 6y ago
      Oh, I know. There's that, and File::Slurp, and a bunch of other stuff one would think would exist from the start, but for some reason don't. But those are all external modules that somebody had to write, that didn't exist at some point in time, and which sometimes one can't use because in some cases you can only rely in what's in core. I'm just wondering at that bizarrely enough, Larry Wall (or somebody else) found it useful to have a simple, convenient way to execute a command with exact arguments, but not for when you need its output. I must have written the same wrapper around open() several dozen times by now, due to needing to write scripts for cases where installing dependencies is undesirable.
      - dolmen 6y ago
        > I'm just wondering at that bizarrely enough, Larry Wall (or somebody else) found it useful to have a simple, convenient way to execute a command with exact arguments, but not for when you need its output. It had backquotes from the start. So it was as convenient (and as unsafe) as the Bourne shell. Of course you quickly want more safety and backquotes are just legacy you can never use for serious stuff.
    - deleted 6y ago
      [deleted]
  - arnsholt 6y ago
    `open my $fh, '-|', 'ls', '-l', $dir` should do the trick I think, but apparently it only works on platforms with a "real fork" (so fine as long as you don't care about Windows, probably).
    - 6y ago
- TeMPOraL 6y ago
  Or CMake. I can excuse shell languages because they're optimizing for being fast to type in a command line, while also being used for writing programs. But other than that, I weep when I think of the time being wasted on languages with arcane syntax. I wish the world would just switch over to s-expressions. I understand that many people will find well thought out syntax cleaner than writing trees. But what languages like Bash or CMake, and arguably JS too, have in common is that their syntax wasn't thought out, it grew organically into a mess. S-expressions eliminate this entire class of problems up front.
znpy 6y ago
I first learned about this hack while reading a debian init script. Thankfully systemd has removed the need about this!
- gorgoiler 6y ago
  Init scripts are just executable files. They can be written in any compiled language you like or even an interpreted language, if you are prepared to add the dependency of an interpreter for that language at boot time. This is a lot simpler than migrating to (and getting locked in to) systemd.
  - otabdeveloper4 6y ago
    systemd is exactly the boot time interpreter you're asking for.
    - corty 6y ago
      No, systemd doesn't really interpret much, unit files are mostly not a full replacement for what init-scripts were capable of. Which leads to lots of Exec=/some/shell/script.sh in unit files which is worse than before because now you have multiple files and more strange interactions.
      - BenjiWiebe 6y ago
        Is that actually common? I write quite a few systemd unit files, and have yet to need a wrapper shell script. And the vast majority, or maybe even all, of the unit files my system has by default are also not calling shell scripts.
        alerighi 6y ago
        Depends on what you are doing. Init scripts where simpler because the boot process was well defined, meaning that you had all your scripts in rc.d that were executed in that particular order. Now with systemd is more difficult to have something start after something else, you have all the dependencies and if you get them wrong it will not work. Worse, the boot process is no longer deterministic, meaning that 99 times it could work and it could break the 100th time. On the other side, systemd is a necessary evil for a modern desktop system, where you have multiple events that could fire (hotplug, network, power events, etc). But not so sure if it's that necessary for a server or an embedded device.
chubot 6y ago
Related: Problems With the test Builtin: What Does -a Mean? http://www.oilshell.org/blog/2017/08/31.html http://www.oilshell.org/blog/2017/08/31.html The POSIX spec did indeed make things cleaner; the last section quotes it and gives some style advice.
dataflow 6y ago
It's still relevant if you use -a, -o, or parentheses (which I guess is reason to avoid them): $ a=1; b=1; c='!'; d='!' $ [ "$a" = "$b" -a "$c" = "$d" ] -bash: [: too many arguments $ [ '!' '(' "$c" = "$d" ')' ] -bash: [: `)' expected, found ! $ echo "${BASH_VERSION}" 5.1.4(1)-release Note that it's not even a bug here; it's necessary for disambiguation.
- arthur2e5 6y ago
  The post mentions the issue is fixed by POSIX, which… notably does not standarize -o -a and parens. I guess now we know why POSIX didn't standardize those.
  - hvdijk 6y ago
    POSIX did standardize those, but the functionality is marked XSI, so only XSI-conforming implementations are required to provide it. There is never a reason to use them, though. They can be replaced by the shell's && and || operators, which nicely bypasses the parsing ambiguities at the same time.
    - dataflow 6y ago
      > There is never a reason to use them, though. They can be replaced by the shell's && and || operators, which nicely bypasses the parsing ambiguities at the same time. Not sure I agree. The equivalent of more complicated expressions like (a AND b) OR (c AND d) with && and || isn't always so "nice" IMO. e.g., try it for this: [ -f a -a -f b -o -f c -a -f d ] I think the simplest syntax will require you to spawn subshells, which is neither particularly nice, nor particularly performant (especially on Windows).
      - sltkr 6y ago
        Is this really so much worse: { test -f a && test -f b; } || { test -f c && test -f d; } ? Yes, it's more verbose, but the original expression isn't particularly readable either, and if you'd want to evaluate something like (a OR b) AND (c OR d) you'd need to introduce parentheses anyway.
kiwijamo 6y ago
If you wish to support the author of ShellCheck, you can become a GitHub sponsor: https://github.com/sponsors/koalaman https://github.com/sponsors/koalaman Disclaimer: I am one of the current sponsors.
- spockz 6y ago
  The GitHub docs speak of being able to do single donations but I’ve not seen the option yet. Is it something you need to explicitly configure as the one being sponsored?
  - koala_man 6y ago
    One-time donations were not originally an option, but I'm happy to see it's been added since. It's now enabled. Thanks!
- dcminter 6y ago
  Thanks for that tip, I've benefitted from Shellcheck a bunch of times so I've just sponsored and wouldn't have without your comment. I've puttered along for years writing little bash scripts for one thing or another and always had the uneasy feeling that I was littering them with errors. Shellcheck's linting showed me (a) that I was right and (b) how to fix them. It's made me a better developer and I'm so grateful for it!
- koala_man 6y ago
  Thank you for your support ^____^
conradludgate 6y ago
Any time I have to do any serious shell scripting at work, I die a little inside. I do need to try out shellcheck thoroughly, it might catch some bugs in our pipelines
- dcminter 6y ago
  I started using it about a year ago (I wasn't aware of it before that) and it's astonishing the number of things it can pick up on in even a five line script. Often they're things you will get away with most or even all of the time in a specific scenario, but there they lurk, waiting to blow your leg off when you start including spaces in a filename or somesuch!
- dataflow 6y ago
  Honestly, the only practical solution to this in my experience is to just buckle up and spend the hours/days it takes to learn the dirty corners of Bash properly. This doesn't mean you have to learn 100% of it, but it does mean you need to learn enough of it that will (a) serve your needs comfortably (instead of making you constantly have to hack around), and (b) give you insight as to what subset(s) to avoid unless absolutely necessary. Same goes with other tools, whether with warranted or unwarranted complexity (like git, Bazel, C++, etc.). Yes it's painful while you're still learning, but it's an investment that makes you more productive and just makes everything so much more tolerable afterward. If you never spend the time then you'll keep dying inside every time you have to deal with tools you don't like.
  - doctoboggan 6y ago
    I feel like I’m overdue for learning she’ll scripting properly. I’ve hacked around for years google as I go. What resources would you suggest for decent training material?
    - dataflow 6y ago
      I wish I had a great answer, but I don't. Maybe these will help, though. What I've done is, every time I came across something that didn't behave the way I expected, I tried to Google it and tinker around with it until I could understand exactly what is going on, so that I "got" the underlying feel for the language and didn't have to look up every consequence every time. e.g., the difference between [ ] and [[ ]], which is confusing at first, becomes more intuitive once you realize [ is a portable POSIX command whereas [[ is special Bash syntax, and once you grasp this, you can understand what its implications are. e.g., that Bash has no way to detect '=' as a special token inside [ ], and consequently you should worry about ambiguities. (This does assume a little background knowledge of parsing ambiguities; if you haven't taken a course related to programming languages, going through one might also be a worthwhile investment.) Also, you have to be on the lookout for things that you would want to do in a different language, and learn how to do it in Bash, like creating functions & local variables. It's mostly a matter of being proactive about learning how to structure your code like you would in other languages. Often I see people try to keep things "simple" by just copy/pasting code so that they don't have to organize things at a higher level. You wouldn't do that in other languages; don't do that in Bash. Some pitfalls you really just do have to learn the hard way; there's no other way I can think of. The key here is to spend the extra minutes necessary to actually learn what broke when you come across it, instead of just tinkering with your code until it works. Like when you realize that (set -e; false && false; echo hi) prints 'hi' despite the fact that (set -e; false; echo hi) does not (this is absolutely bonkers), dig in and figure out what's going on so you don't shoot yourself in the foot the next time. You can't avoid it the first time, but you can at least make sure to avoid it in the future. Also, always try to think of unintended consequences when you learn how to do something in Bash. Like when you learn how `trap 'rm -f file.bak' EXIT` lets you clean up a file, ask yourself what happens if someone has already set a trap? Ask yourself how to dynamically generate and pass a file name to that command safely? Ask yourself in what scope/context the command is evaluated? Spend some time figuring these out either online or by tinkering manually, then remember your solution for later (maybe write it down). Possibly the most costly mistake you can make is to assume something will work in exactly the same way you expect in every other language, because chances are pretty darn good it won't. Even pipelines (which you probably already use) are tricky if you start thinking about them... e.g., what happens if the first command terminates prematurely? What happens if you assign to a variable? etc. Some sites I can recommend for some things: - https://pubs.opengroup.org/onlinepubs/9699919799/utilities/sh.html https://pubs.opengroup.org/onlinepubs/9699919799/utilities/s... - https://tldp.org/LDP/abs/html/index.html https://tldp.org/LDP/abs/html/index.html - https://mywiki.wooledge.org/BashPitfalls https://mywiki.wooledge.org/BashPitfalls The POSIX pages (OpenGroup, first link) are useful for understanding what you can rely on; I'd use that as a command reference whenever it's possible. In some cases they're overly pessimistic (e.g., you can probably rely on 'local' despite it not being POSIX), but generally they're good starting points. tldp is where I still look up some things, like the heredoc syntax. I've probably looked that up 20 times and I still don't remember it well enough for some reason. And it might be fun to just browse through the BashPitfalls page once you have some comfort with shell scripting just to get an idea of what can go wrong, but I wouldn't really recommend it as a learning guide, and you definitely shouldn't expect to memorize all of it.
deleted 6y ago
[deleted]
nwellnhof 6y ago
The "POSIX prescriptions" mentioned in the quoted comment can be found here under "algorithm for determining the precedence": https://pubs.opengroup.org/onlinepubs/9699919799/utilities/test.html https://pubs.opengroup.org/onlinepubs/9699919799/utilities/t...
rustshellscript 6y ago
The only good part of sh/bash script is to run commands with piping/redirection support IMO，other than that I saw posts like this to explain sh/bash script’s puzzles/pitfalls all the time. Maybe it is time to consider using rust instead: https://github.com/rust-shell-script/rust_cmd_lib https://github.com/rust-shell-script/rust_cmd_lib ?
- beltsazar 6y ago
  I'm not sure why you're downvoted, but thanks for the link! Coincidentally, I've been searching this kind of library in Rust for the past few days.
majkinetor 6y ago
Amazing how people still put up with this nonsense. Bash really has no place in this age.
- skywhopper 6y ago
  Bash has multiple better solutions to this. This is a really really really old hack. New code that uses it is either trying to be backwards compatible to the extreme or is more likely just someone unthinkingly copying what they’ve been taught or seen.
- qiqitori 6y ago
  What's a decent replacement shell? Sometimes I think there should be one, with all backwards compatibility for if/looping thrown out, but still adhering to the core concept of mostly being made of a handful of builtins and otherwise executing commands (i.e., without having to type system("") or exec("") etc. to execute commands).
  - mkj 6y ago
    Python with sh.py is often alright in the place of a shell script.
    - majkinetor 6y ago
      Really ? sh.ls("-l", "/tmp", color="never")
  - majkinetor 6y ago
    PowerShell obvioslly, if you can handle awesome. Now there will be tones of angry dudes complaining about: - Optional verbosity - 1-5s startup time on some machines - Random MS agenda to conquer and destroy - Random problem with usage that is solved by RTFM - Suggesting python, ruby, go, node, perl and whatever non-shell language Thankfully, ignoring is still a great human feature.
    - jen20 6y ago
superasn 6y ago
I sometimes wonder why bash transpilers aren't more popular? Bash is such a strange language (or maybe just very old with lots of cruft) that it's impossible to write anything meaningful without check SO 5 times. Given i haven't spent a lot of time writing shell script yet but the syntax is just super weird, like I recently learned(1) that when you When you write if [ -e /etc/passwd ]; then .. that bracket is not shell syntax but just a regular command with a funny name. Is there a recommended bash transpiler like what babel is to Js? So i can write bash scripts using a more coherent language and it spits out .sh files. (1) https://news.ycombinator.com/item?id=26701079 https://news.ycombinator.com/item?id=26701079
- gspr 6y ago
  I wouldn't go that far as calling it impossible, but it sure is crufty and quirky seen with modern eyes. My "favorite" (i.e. most terrifying) BASH fact is this: https://unix.stackexchange.com/questions/121013/how-does-linux-deal-with-shell-scripts https://unix.stackexchange.com/questions/121013/how-does-lin...
  - codetrotter 6y ago
    Interesting that the answer to that question names the file as .bash – I do this too, but I didn’t know anyone else did. I do this because my shell scripts use bash specific features, and I want to distinguish them from portable shell scripts that use POSIX shell features only. The latter being the only files that really deserve being named as .sh in my eyes :)
- Hendrikto 6y ago
  > I recently learned(1) that when you When you write if [ -e /etc/passwd ]; then .. that bracket is not shell syntax but just a regular command with a funny name. That’s how it used to be. Nowadays, most of the time it is a built-in, not it‘s own executable anymore.
- devit 6y ago
  That's false, [ is a bash shell builtin (it's also a coreutils program, but the program is not called unless you specify /usr/bin/[ or use a shell that doesn't have it built-in).
tragomaskhalos 6y ago
I use this hack as a matter of course, having internalised it aeons ago. Bash does lots of things better now but as others have commented it is still so fundamentally weird that an innate conservatism is often a wise approach, especially when one lacks the time (or motivation) to explore its "newer" features.
p4l4g4 6y ago
Wondered for ages whether there is a real situation where this is needed. Good to see this publicly debunked. Now I have a resource to point to when I get the "better safe than sorry" argument.
- DonHopkins 6y ago
  What's been publicly and thoroughly debunked is the very idea of writing shell scripts in the first place, instead of using a non-ridiculous scripting language like Python, Perl, or JavaScript. If you want to be safe instead of sorry, then never write shell scripts.
hibbelig 6y ago
I thought it was about empty strings... Going to check that
- patal 6y ago
  The article only considers quoted arguments. Quoting should be recommended for test, and OP probably even has a formal check for quotes being in place. However, let's just set that aside for now. Quotes are optional. And if you don't put quotes, then yes, you need to use `[ x$var = xval ]` to address empty variables.
  - tinus_hn 6y ago
    I wonder what happens when $var starts with ‘-‘, it wouldn’t surprise me if it’s taken as an option.
    - dolmen 6y ago
      It's time to read the article.
      - tinus_hn 6y ago
        Ah, upon rereading it turns out it is still an issue: ... would fail to run test commands whose left-hand side matched a unary operator. This was fixed [...] in 1979. However, test and [ were also available as separate executables, and appear to have retained a variant of the buggy behavior:
eb0la 6y ago
I learnt this trick from OpenBSD 2.5 (yes, a long time ago). I still remember getting root with a shell-script mp3 player that a friend coded because of this. The exploit was based in inserting some \$$ make_a_new_root_account_in_etcpassw $ inside the variable inside the test. As a result, both my friend and I stopped writing shell scripts as CGI and started learning perl just to do it.
montroser 6y ago
Note that this is a relic of the past, as TFA explains. Modern bash can be totally manageable and even fun, if you take the time to learn and have a good style guide and shellcheck by your side.
laci37 6y ago
My second task at my first job was to connect up a horrible mess of bash, sed, awk, python and R to use a web service as input instead of local files. I ended up rewriting the whole thing to python. It was a good decision.
donatj 6y ago
In my book with zsh being a very popular shell and it having been a bug as little as 5 years ago, I’d almost learn towards recommending it rather than recommending against? It seems like a relatively low impact fix that improves portability.
raldi 6y ago
Seeing or typing x$foo is a reminder that shell scripts are unsuited for anything more than a few lines long. If you're tempted to switch to [[ ]], just switch all the way to a real programming language.
adrianmonk 6y ago
On a side note, I think this style tweak makes it more readable: [ x"$var" = x"val" ] My eye doesn't have to separate "xval" into "x" and "val", so it's more obvious that the value being compared against is "val". (It behaves the same way since the shell allows quoting to start in the middle.)
SavantIdiot 6y ago
"This happened because the utility used a simple recursive descent parser without backtracking, which gave unary operators precedence over binary operators and ignored trailing arguments." It is a bit sad that his problem has to be encountered and resolved over and over and over again. Granted, it was a SIMPLE PARSER because resources were extremely tight. Note, I don't mean shells, I mean working with projects that implement parsers. As soon as someone busts out lex/yacc I know I'm going to see at least one "ah, we didn't think of that" issue. Parsers are hard.
- rspeele 6y ago
  While I agree that there can be pitfalls to parsing, most of the time it isn't as bad as this. The "test" or "[" program has a unique challenge in that it never gets to see the quotation marks or variable references, because the shell already expanded them. So when test sees that one of its arguments is the string "-f", it has no way to know whether that string came from a variable like $var, a quoted string literal like "-f", or just the text -f. Most of the time if you're writing a parser for your own DSL you don't have this problem. You can tell the difference between the string literal "+" and the operator +.
chunkyks 6y ago
So, in conclusion it's a hack... but, it's also incredibly robust. It doesn't hurt, and we acknowledge there are cases where it could help. It's idiomatic enough that literally everyone who's done a nontrivial amount of scripting is familiar with it. At some point, "decades old idiomatic hack that increases robustness" stops being a hack and becomes an idiomatic way to increase robustness. A better conclusion would be "yes, keep using it".
- m463 6y ago
  > So, in conclusion it's a hack... but, it's also incredibly robust. You know, shell scripts are a hack. I really wish there was a middle ground between shell scripts and say python. Python has proper lists and hashes and loops and can manipulate paths with spaces or quotes or unicode. It doesn't get dragged down by the escape-an-escape-within-a-regex nonsense that makes things undecipherable. or ${VAR:-foo} or ${VAR##ugh} But python is a bit fumbly when it comes to invoking external programs. try: ... subprocess.bleh except: ... ugh.
  - zokier 6y ago
    Perl?
  - acdha 6y ago
    > But python is a bit fumbly when it comes to invoking external programs. try: ... subprocess.bleh except: ... ugh. I actually tend to prefer that these days — if I write shell I'm tossing in "set -e -u -o pipefail" anyway and subprocess's newer APIs end up being cleaner when you need to do anything non-trivial for error handling (I usually end up with one `try:` block for most of the program and maybe one or two `except: cleanup(); raise` blocks. Having the logic be clean and consistent saves you so much time when you revisit that program a month later. The biggest wins I've seen from quoting external program arguments — either not needing to do so at all or passing things cleanly through shlex.quote makes it so much easier when you have the possibility of any special characters in arguments.
  - chubot 6y ago
    That's one good way to describe Oil: a middle ground between shell and Python. e.g. this post has the slogan that it's "for Python and JS programmers who avoid shell". http://www.oilshell.org/blog/2020/01/simplest-explanation.html http://www.oilshell.org/blog/2020/01/simplest-explanation.ht... Oil has hash tables, and can manipulate paths with spaces -- in fact the latest blog post is about that! http://www.oilshell.org/blog/2021/04/simple-word-eval.html http://www.oilshell.org/blog/2021/04/simple-word-eval.html Also the same blog post mentions that we should get rid of ${VAR##ugh}. Help wanted :)
aasasd 6y ago
As usual, the problem is that Sh & co. use in-band control, especially when talking to utilities. Just a mess of text being passed around, of which every program must make sense on its own—but this happening in shell's own functions takes the cake. Moreover, similar problems are inherited by some programs that try to do shell-like commands but put their own syntax in the way. E.g. Ansible, where YAML, JSON, module call DSL and Jinja are all mixed on top of the commands so you can bump into any of them.
- improv32 6y ago
  Interestingly enough, this isn't a mess in the shell's own builtins. The [ ] expression syntax is provided by the program /usr/bin/[ which is (often) a symlink to /usr/bin/test. It's just a clever trick! [ is a program that evaluates it's argv as an an expression, less the trailing ], and then returns the result as an error code. Actually re-reading your post you probably know that already! Nevermind then. Maybe someone reading this will be enlightened
  - aasasd 6y ago
    Yes, I know about `[` being both a program and (potentially) a built-in. However the problem is exactly that an Sh-style shell can't easily divorce itself from the same gotchas unless it invents a DSL for its builtins with different semantics—because normally it must parse vars before invoking the command, and thus vars are interpolated into the text, which the command is. If the shell does the sane thing of interpreting the command before evaluating vars, it means breaking off from the normal semantics and thus being inconsistent. I imagine the introduction of `[[` and `((` in Bash may have come partly from this reasoning (to abandon the POSIX semantics for these commands), though I never delved far enough into nerdery to learn the difference from `[`—so don't know if they do evaluate vars in the sensible way. I keep wondering if there are any shells or other software calling into utils, that manage to walk the edge of writing commands without bumping into such syntax problems—but avoid having to "enquote" "each" "argument" all the time.
- naniwaduni 6y ago
  [ is not, in principle, the shell's own function. It merely happens to be implemented as one in most shells anyone actually uses as an optimization.
mikedilger 6y ago
I still use this hack without thinking about it in case the variable on the left is empty. But I suppose I should actually be quoting. [ $X == "hello" ] When X is empty, it gives: example.sh: line 5: [: ==: unary operator expected So I fix it (apparently wrongly) with: [ x$X == x"hello" ] I'll make a point of doing this instead: [ "$X" == "hello" ]
- koala_man 6y ago
  Definitely quote. `[ x$X == x"hello" ]` may fail for a variety of different values of X like `*`, `Rocky_[1976].jpg`, `hello world` and `foo = xfoo -o bar` Quoting works in all those cases.
  - naniwaduni 6y ago
    Of the listed, it should only actually fail on the ones with spaces in them (or rather, the ones that get split according to IFS).
    - koala_man 6y ago
      Whether or not it should, it may fail for any kind of glob. This could be either because the glob ends up unintentionally matching something: $ var='*'; touch xfoo xbar $ [ x$var == x"hello" ] bash: [: too many arguments $ var="Rocky_[1976].jpg"; touch xRocky_7.jpg $ [ x$var == x"Rocky_[1976].jpg" ] (false, should be true) or because you have something like nullglob or failglob enabled: $ var="Rocky_[1976].jpg"; shopt -s nullglob $ [ x$var == x"hello" ] bash: [: ==: unary operator expected
wruza 6y ago
To me the question always was what’s the point of [ba]sh for non-interactive shell scripting, when there is perl (I’m aware that it has issues with versions, portability to non-PCs, etc, so think “theoretical standardized lightweight perl mode for sh”, not a real one). Shell syntax and semantics are a pile of ugly hacks and landmines, which were done because everyone wanted to program cli. When you want to program, just take a programming language, not a poor excuse of it. The amount of time you end up wasting with bash is much greater than reading perldoc *, and mastering bash never returns an investment.
jude- 6y ago
People complaining about how underpowered shells are should have a look at Ion [1]. It's mostly the same as bash, but with first-class support for arrays, maps, byte slices, variable scoping, and (primitive) type-checking. Oh, and it's written in Rust. [1] https://gitlab.redox-os.org/redox-os/ion https://gitlab.redox-os.org/redox-os/ion
- svnpenn 6y ago
  Last release was 3 years ago...
  - jude- 6y ago
    Last commit was 2 weeks ago.
- chubot 6y ago
  BTW Ion is mentioned here along with many other alternative shells: https://github.com/oilshell/oil/wiki/Alternative-Shells https://github.com/oilshell/oil/wiki/Alternative-Shells
deleted 6y ago
[deleted]

414 points by JNRowe 6y ago

> Amazingly, the x-hack could be used to work around certain bugs all the way up until 2015, seven years after StackOverflow wrote it off as an archaic relic of the past! Oof. If people think JS is bad wait until they try to do anything moderately complex in shell script

iso1631 6y ago
> If people think JS is bad wait until they try to do anything moderately complex in shell script The number of times I've started with a simple bash script and regretted it down the line. Now I refactor into perl far earlier in the process
dale_glass 6y ago
Yup. These days if it's more than a few lines, I write it in Perl instead if it's for personal use, or Python if it's going to be shared (Python just takes me more effort still). For all the vaunted Unix Philosophy it's amazing how clunky some of it is. Countless shell scripts still trip over spaces in filenames. Of those that don't, nearly all of them will still be confused by any unusual characters like newlines. If you want something done properly, you have to pull out a proper scripting language, to have readdir() and the ability to pass arguments directly to a process. Even Perl, which generally excels in such tasks has weird lapses in convenience. You can run a command with one line of code without the possibility of confusion with `system("ls", "-l", $dir)`, but you can't get its output that way. There's no version of `system` that'd allow you to both explicitly specify each argument to the process, and obtain its output. You either use backticks and risk quoting trouble, or need to use `open`, which is a lot more verbose. It's interesting that it took Microsoft to do a new approach in this regard. PowerShell has its amount of weirdness I really hate, such as that the environment escapes the program into the commandline, but it's really refreshing how it dispenses with brittle grep, awk and cut stuff.
- mfontani 6y ago
  For Perl, you might want to look at https://metacpan.org/pod/IPC::Run3 https://metacpan.org/pod/IPC::Run3 You can: - specify each argument separately - get separate stdout and stderr - provide input
  - dale_glass 6y ago
    Oh, I know. There's that, and File::Slurp, and a bunch of other stuff one would think would exist from the start, but for some reason don't. But those are all external modules that somebody had to write, that didn't exist at some point in time, and which sometimes one can't use because in some cases you can only rely in what's in core. I'm just wondering at that bizarrely enough, Larry Wall (or somebody else) found it useful to have a simple, convenient way to execute a command with exact arguments, but not for when you need its output. I must have written the same wrapper around open() several dozen times by now, due to needing to write scripts for cases where installing dependencies is undesirable.
    - dolmen 6y ago
      > I'm just wondering at that bizarrely enough, Larry Wall (or somebody else) found it useful to have a simple, convenient way to execute a command with exact arguments, but not for when you need its output. It had backquotes from the start. So it was as convenient (and as unsafe) as the Bourne shell. Of course you quickly want more safety and backquotes are just legacy you can never use for serious stuff.
  - deleted 6y ago
    [deleted]
- arnsholt 6y ago
  `open my $fh, '-|', 'ls', '-l', $dir` should do the trick I think, but apparently it only works on platforms with a "real fork" (so fine as long as you don't care about Windows, probably).
  - 6y ago
TeMPOraL 6y ago
Or CMake. I can excuse shell languages because they're optimizing for being fast to type in a command line, while also being used for writing programs. But other than that, I weep when I think of the time being wasted on languages with arcane syntax. I wish the world would just switch over to s-expressions. I understand that many people will find well thought out syntax cleaner than writing trees. But what languages like Bash or CMake, and arguably JS too, have in common is that their syntax wasn't thought out, it grew organically into a mess. S-expressions eliminate this entire class of problems up front.

znpy 6y ago

I first learned about this hack while reading a debian init script. Thankfully systemd has removed the need about this!

gorgoiler 6y ago
Init scripts are just executable files. They can be written in any compiled language you like or even an interpreted language, if you are prepared to add the dependency of an interpreter for that language at boot time. This is a lot simpler than migrating to (and getting locked in to) systemd.
- otabdeveloper4 6y ago
  systemd is exactly the boot time interpreter you're asking for.
  - corty 6y ago
    No, systemd doesn't really interpret much, unit files are mostly not a full replacement for what init-scripts were capable of. Which leads to lots of Exec=/some/shell/script.sh in unit files which is worse than before because now you have multiple files and more strange interactions.
    - BenjiWiebe 6y ago
      Is that actually common? I write quite a few systemd unit files, and have yet to need a wrapper shell script. And the vast majority, or maybe even all, of the unit files my system has by default are also not calling shell scripts.
      - alerighi 6y ago
        Depends on what you are doing. Init scripts where simpler because the boot process was well defined, meaning that you had all your scripts in rc.d that were executed in that particular order. Now with systemd is more difficult to have something start after something else, you have all the dependencies and if you get them wrong it will not work. Worse, the boot process is no longer deterministic, meaning that 99 times it could work and it could break the 100th time. On the other side, systemd is a necessary evil for a modern desktop system, where you have multiple events that could fire (hotplug, network, power events, etc). But not so sure if it's that necessary for a server or an embedded device.

chubot 6y ago

Related: Problems With the test Builtin: What Does -a Mean? http://www.oilshell.org/blog/2017/08/31.html http://www.oilshell.org/blog/2017/08/31.html The POSIX spec did indeed make things cleaner; the last section quotes it and gives some style advice.

dataflow 6y ago

It's still relevant if you use -a, -o, or parentheses (which I guess is reason to avoid them): $ a=1; b=1; c='!'; d='!' $ [ "$a" = "$b" -a "$c" = "$d" ] -bash: [: too many arguments $ [ '!' '(' "$c" = "$d" ')' ] -bash: [: `)' expected, found ! $ echo "${BASH_VERSION}" 5.1.4(1)-release Note that it's not even a bug here; it's necessary for disambiguation.

arthur2e5 6y ago
The post mentions the issue is fixed by POSIX, which… notably does not standarize -o -a and parens. I guess now we know why POSIX didn't standardize those.
- hvdijk 6y ago
  POSIX did standardize those, but the functionality is marked XSI, so only XSI-conforming implementations are required to provide it. There is never a reason to use them, though. They can be replaced by the shell's && and || operators, which nicely bypasses the parsing ambiguities at the same time.
  - dataflow 6y ago
    > There is never a reason to use them, though. They can be replaced by the shell's && and || operators, which nicely bypasses the parsing ambiguities at the same time. Not sure I agree. The equivalent of more complicated expressions like (a AND b) OR (c AND d) with && and || isn't always so "nice" IMO. e.g., try it for this: [ -f a -a -f b -o -f c -a -f d ] I think the simplest syntax will require you to spawn subshells, which is neither particularly nice, nor particularly performant (especially on Windows).
    - sltkr 6y ago
      Is this really so much worse: { test -f a && test -f b; } || { test -f c && test -f d; } ? Yes, it's more verbose, but the original expression isn't particularly readable either, and if you'd want to evaluate something like (a OR b) AND (c OR d) you'd need to introduce parentheses anyway.

kiwijamo 6y ago

If you wish to support the author of ShellCheck, you can become a GitHub sponsor: https://github.com/sponsors/koalaman https://github.com/sponsors/koalaman Disclaimer: I am one of the current sponsors.

spockz 6y ago
The GitHub docs speak of being able to do single donations but I’ve not seen the option yet. Is it something you need to explicitly configure as the one being sponsored?
- koala_man 6y ago
  One-time donations were not originally an option, but I'm happy to see it's been added since. It's now enabled. Thanks!
dcminter 6y ago
Thanks for that tip, I've benefitted from Shellcheck a bunch of times so I've just sponsored and wouldn't have without your comment. I've puttered along for years writing little bash scripts for one thing or another and always had the uneasy feeling that I was littering them with errors. Shellcheck's linting showed me (a) that I was right and (b) how to fix them. It's made me a better developer and I'm so grateful for it!
koala_man 6y ago
Thank you for your support ^____^

conradludgate 6y ago

Any time I have to do any serious shell scripting at work, I die a little inside. I do need to try out shellcheck thoroughly, it might catch some bugs in our pipelines

dcminter 6y ago
I started using it about a year ago (I wasn't aware of it before that) and it's astonishing the number of things it can pick up on in even a five line script. Often they're things you will get away with most or even all of the time in a specific scenario, but there they lurk, waiting to blow your leg off when you start including spaces in a filename or somesuch!
dataflow 6y ago
Honestly, the only practical solution to this in my experience is to just buckle up and spend the hours/days it takes to learn the dirty corners of Bash properly. This doesn't mean you have to learn 100% of it, but it does mean you need to learn enough of it that will (a) serve your needs comfortably (instead of making you constantly have to hack around), and (b) give you insight as to what subset(s) to avoid unless absolutely necessary. Same goes with other tools, whether with warranted or unwarranted complexity (like git, Bazel, C++, etc.). Yes it's painful while you're still learning, but it's an investment that makes you more productive and just makes everything so much more tolerable afterward. If you never spend the time then you'll keep dying inside every time you have to deal with tools you don't like.
- doctoboggan 6y ago
  I feel like I’m overdue for learning she’ll scripting properly. I’ve hacked around for years google as I go. What resources would you suggest for decent training material?
  - dataflow 6y ago
    I wish I had a great answer, but I don't. Maybe these will help, though. What I've done is, every time I came across something that didn't behave the way I expected, I tried to Google it and tinker around with it until I could understand exactly what is going on, so that I "got" the underlying feel for the language and didn't have to look up every consequence every time. e.g., the difference between [ ] and [[ ]], which is confusing at first, becomes more intuitive once you realize [ is a portable POSIX command whereas [[ is special Bash syntax, and once you grasp this, you can understand what its implications are. e.g., that Bash has no way to detect '=' as a special token inside [ ], and consequently you should worry about ambiguities. (This does assume a little background knowledge of parsing ambiguities; if you haven't taken a course related to programming languages, going through one might also be a worthwhile investment.) Also, you have to be on the lookout for things that you would want to do in a different language, and learn how to do it in Bash, like creating functions & local variables. It's mostly a matter of being proactive about learning how to structure your code like you would in other languages. Often I see people try to keep things "simple" by just copy/pasting code so that they don't have to organize things at a higher level. You wouldn't do that in other languages; don't do that in Bash. Some pitfalls you really just do have to learn the hard way; there's no other way I can think of. The key here is to spend the extra minutes necessary to actually learn what broke when you come across it, instead of just tinkering with your code until it works. Like when you realize that (set -e; false && false; echo hi) prints 'hi' despite the fact that (set -e; false; echo hi) does not (this is absolutely bonkers), dig in and figure out what's going on so you don't shoot yourself in the foot the next time. You can't avoid it the first time, but you can at least make sure to avoid it in the future. Also, always try to think of unintended consequences when you learn how to do something in Bash. Like when you learn how `trap 'rm -f file.bak' EXIT` lets you clean up a file, ask yourself what happens if someone has already set a trap? Ask yourself how to dynamically generate and pass a file name to that command safely? Ask yourself in what scope/context the command is evaluated? Spend some time figuring these out either online or by tinkering manually, then remember your solution for later (maybe write it down). Possibly the most costly mistake you can make is to assume something will work in exactly the same way you expect in every other language, because chances are pretty darn good it won't. Even pipelines (which you probably already use) are tricky if you start thinking about them... e.g., what happens if the first command terminates prematurely? What happens if you assign to a variable? etc. Some sites I can recommend for some things: - https://pubs.opengroup.org/onlinepubs/9699919799/utilities/sh.html https://pubs.opengroup.org/onlinepubs/9699919799/utilities/s... - https://tldp.org/LDP/abs/html/index.html https://tldp.org/LDP/abs/html/index.html - https://mywiki.wooledge.org/BashPitfalls https://mywiki.wooledge.org/BashPitfalls The POSIX pages (OpenGroup, first link) are useful for understanding what you can rely on; I'd use that as a command reference whenever it's possible. In some cases they're overly pessimistic (e.g., you can probably rely on 'local' despite it not being POSIX), but generally they're good starting points. tldp is where I still look up some things, like the heredoc syntax. I've probably looked that up 20 times and I still don't remember it well enough for some reason. And it might be fun to just browse through the BashPitfalls page once you have some comfort with shell scripting just to get an idea of what can go wrong, but I wouldn't really recommend it as a learning guide, and you definitely shouldn't expect to memorize all of it.

deleted 6y ago

[deleted]

nwellnhof 6y ago

The "POSIX prescriptions" mentioned in the quoted comment can be found here under "algorithm for determining the precedence": https://pubs.opengroup.org/onlinepubs/9699919799/utilities/test.html https://pubs.opengroup.org/onlinepubs/9699919799/utilities/t...

rustshellscript 6y ago

The only good part of sh/bash script is to run commands with piping/redirection support IMO，other than that I saw posts like this to explain sh/bash script’s puzzles/pitfalls all the time. Maybe it is time to consider using rust instead: https://github.com/rust-shell-script/rust_cmd_lib https://github.com/rust-shell-script/rust_cmd_lib ?

beltsazar 6y ago
I'm not sure why you're downvoted, but thanks for the link! Coincidentally, I've been searching this kind of library in Rust for the past few days.

majkinetor 6y ago

Amazing how people still put up with this nonsense. Bash really has no place in this age.

skywhopper 6y ago
Bash has multiple better solutions to this. This is a really really really old hack. New code that uses it is either trying to be backwards compatible to the extreme or is more likely just someone unthinkingly copying what they’ve been taught or seen.
qiqitori 6y ago
What's a decent replacement shell? Sometimes I think there should be one, with all backwards compatibility for if/looping thrown out, but still adhering to the core concept of mostly being made of a handful of builtins and otherwise executing commands (i.e., without having to type system("") or exec("") etc. to execute commands).
- mkj 6y ago
  Python with sh.py is often alright in the place of a shell script.
  - majkinetor 6y ago
    Really ? sh.ls("-l", "/tmp", color="never")
- majkinetor 6y ago
  PowerShell obvioslly, if you can handle awesome. Now there will be tones of angry dudes complaining about: - Optional verbosity - 1-5s startup time on some machines - Random MS agenda to conquer and destroy - Random problem with usage that is solved by RTFM - Suggesting python, ruby, go, node, perl and whatever non-shell language Thankfully, ignoring is still a great human feature.
  - jen20 6y ago

superasn 6y ago

I sometimes wonder why bash transpilers aren't more popular? Bash is such a strange language (or maybe just very old with lots of cruft) that it's impossible to write anything meaningful without check SO 5 times. Given i haven't spent a lot of time writing shell script yet but the syntax is just super weird, like I recently learned(1) that when you When you write if [ -e /etc/passwd ]; then .. that bracket is not shell syntax but just a regular command with a funny name. Is there a recommended bash transpiler like what babel is to Js? So i can write bash scripts using a more coherent language and it spits out .sh files. (1) https://news.ycombinator.com/item?id=26701079 https://news.ycombinator.com/item?id=26701079

gspr 6y ago
I wouldn't go that far as calling it impossible, but it sure is crufty and quirky seen with modern eyes. My "favorite" (i.e. most terrifying) BASH fact is this: https://unix.stackexchange.com/questions/121013/how-does-linux-deal-with-shell-scripts https://unix.stackexchange.com/questions/121013/how-does-lin...
- codetrotter 6y ago
  Interesting that the answer to that question names the file as .bash – I do this too, but I didn’t know anyone else did. I do this because my shell scripts use bash specific features, and I want to distinguish them from portable shell scripts that use POSIX shell features only. The latter being the only files that really deserve being named as .sh in my eyes :)
Hendrikto 6y ago
> I recently learned(1) that when you When you write if [ -e /etc/passwd ]; then .. that bracket is not shell syntax but just a regular command with a funny name. That’s how it used to be. Nowadays, most of the time it is a built-in, not it‘s own executable anymore.
devit 6y ago
That's false, [ is a bash shell builtin (it's also a coreutils program, but the program is not called unless you specify /usr/bin/[ or use a shell that doesn't have it built-in).

tragomaskhalos 6y ago

I use this hack as a matter of course, having internalised it aeons ago. Bash does lots of things better now but as others have commented it is still so fundamentally weird that an innate conservatism is often a wise approach, especially when one lacks the time (or motivation) to explore its "newer" features.

p4l4g4 6y ago

Wondered for ages whether there is a real situation where this is needed. Good to see this publicly debunked. Now I have a resource to point to when I get the "better safe than sorry" argument.

DonHopkins 6y ago
What's been publicly and thoroughly debunked is the very idea of writing shell scripts in the first place, instead of using a non-ridiculous scripting language like Python, Perl, or JavaScript. If you want to be safe instead of sorry, then never write shell scripts.

hibbelig 6y ago

I thought it was about empty strings... Going to check that

patal 6y ago
The article only considers quoted arguments. Quoting should be recommended for test, and OP probably even has a formal check for quotes being in place. However, let's just set that aside for now. Quotes are optional. And if you don't put quotes, then yes, you need to use `[ x$var = xval ]` to address empty variables.
- tinus_hn 6y ago
  I wonder what happens when $var starts with ‘-‘, it wouldn’t surprise me if it’s taken as an option.
  - dolmen 6y ago
    It's time to read the article.
    - tinus_hn 6y ago
      Ah, upon rereading it turns out it is still an issue: ... would fail to run test commands whose left-hand side matched a unary operator. This was fixed [...] in 1979. However, test and [ were also available as separate executables, and appear to have retained a variant of the buggy behavior:

eb0la 6y ago

I learnt this trick from OpenBSD 2.5 (yes, a long time ago). I still remember getting root with a shell-script mp3 player that a friend coded because of this. The exploit was based in inserting some \$$ make_a_new_root_account_in_etcpassw $ inside the variable inside the test. As a result, both my friend and I stopped writing shell scripts as CGI and started learning perl just to do it.

montroser 6y ago

Note that this is a relic of the past, as TFA explains. Modern bash can be totally manageable and even fun, if you take the time to learn and have a good style guide and shellcheck by your side.

laci37 6y ago

My second task at my first job was to connect up a horrible mess of bash, sed, awk, python and R to use a web service as input instead of local files. I ended up rewriting the whole thing to python. It was a good decision.

donatj 6y ago

In my book with zsh being a very popular shell and it having been a bug as little as 5 years ago, I’d almost learn towards recommending it rather than recommending against? It seems like a relatively low impact fix that improves portability.

raldi 6y ago

Seeing or typing x$foo is a reminder that shell scripts are unsuited for anything more than a few lines long. If you're tempted to switch to [[ ]], just switch all the way to a real programming language.

adrianmonk 6y ago

On a side note, I think this style tweak makes it more readable: [ x"$var" = x"val" ] My eye doesn't have to separate "xval" into "x" and "val", so it's more obvious that the value being compared against is "val". (It behaves the same way since the shell allows quoting to start in the middle.)

SavantIdiot 6y ago

"This happened because the utility used a simple recursive descent parser without backtracking, which gave unary operators precedence over binary operators and ignored trailing arguments." It is a bit sad that his problem has to be encountered and resolved over and over and over again. Granted, it was a SIMPLE PARSER because resources were extremely tight. Note, I don't mean shells, I mean working with projects that implement parsers. As soon as someone busts out lex/yacc I know I'm going to see at least one "ah, we didn't think of that" issue. Parsers are hard.

rspeele 6y ago
While I agree that there can be pitfalls to parsing, most of the time it isn't as bad as this. The "test" or "[" program has a unique challenge in that it never gets to see the quotation marks or variable references, because the shell already expanded them. So when test sees that one of its arguments is the string "-f", it has no way to know whether that string came from a variable like $var, a quoted string literal like "-f", or just the text -f. Most of the time if you're writing a parser for your own DSL you don't have this problem. You can tell the difference between the string literal "+" and the operator +.

chunkyks 6y ago

So, in conclusion it's a hack... but, it's also incredibly robust. It doesn't hurt, and we acknowledge there are cases where it could help. It's idiomatic enough that literally everyone who's done a nontrivial amount of scripting is familiar with it. At some point, "decades old idiomatic hack that increases robustness" stops being a hack and becomes an idiomatic way to increase robustness. A better conclusion would be "yes, keep using it".

m463 6y ago
> So, in conclusion it's a hack... but, it's also incredibly robust. You know, shell scripts are a hack. I really wish there was a middle ground between shell scripts and say python. Python has proper lists and hashes and loops and can manipulate paths with spaces or quotes or unicode. It doesn't get dragged down by the escape-an-escape-within-a-regex nonsense that makes things undecipherable. or ${VAR:-foo} or ${VAR##ugh} But python is a bit fumbly when it comes to invoking external programs. try: ... subprocess.bleh except: ... ugh.
- zokier 6y ago
  Perl?
- acdha 6y ago
  > But python is a bit fumbly when it comes to invoking external programs. try: ... subprocess.bleh except: ... ugh. I actually tend to prefer that these days — if I write shell I'm tossing in "set -e -u -o pipefail" anyway and subprocess's newer APIs end up being cleaner when you need to do anything non-trivial for error handling (I usually end up with one `try:` block for most of the program and maybe one or two `except: cleanup(); raise` blocks. Having the logic be clean and consistent saves you so much time when you revisit that program a month later. The biggest wins I've seen from quoting external program arguments — either not needing to do so at all or passing things cleanly through shlex.quote makes it so much easier when you have the possibility of any special characters in arguments.
- chubot 6y ago
  That's one good way to describe Oil: a middle ground between shell and Python. e.g. this post has the slogan that it's "for Python and JS programmers who avoid shell". http://www.oilshell.org/blog/2020/01/simplest-explanation.html http://www.oilshell.org/blog/2020/01/simplest-explanation.ht... Oil has hash tables, and can manipulate paths with spaces -- in fact the latest blog post is about that! http://www.oilshell.org/blog/2021/04/simple-word-eval.html http://www.oilshell.org/blog/2021/04/simple-word-eval.html Also the same blog post mentions that we should get rid of ${VAR##ugh}. Help wanted :)

aasasd 6y ago

As usual, the problem is that Sh & co. use in-band control, especially when talking to utilities. Just a mess of text being passed around, of which every program must make sense on its own—but this happening in shell's own functions takes the cake. Moreover, similar problems are inherited by some programs that try to do shell-like commands but put their own syntax in the way. E.g. Ansible, where YAML, JSON, module call DSL and Jinja are all mixed on top of the commands so you can bump into any of them.

improv32 6y ago
Interestingly enough, this isn't a mess in the shell's own builtins. The [ ] expression syntax is provided by the program /usr/bin/[ which is (often) a symlink to /usr/bin/test. It's just a clever trick! [ is a program that evaluates it's argv as an an expression, less the trailing ], and then returns the result as an error code. Actually re-reading your post you probably know that already! Nevermind then. Maybe someone reading this will be enlightened
- aasasd 6y ago
  Yes, I know about `[` being both a program and (potentially) a built-in. However the problem is exactly that an Sh-style shell can't easily divorce itself from the same gotchas unless it invents a DSL for its builtins with different semantics—because normally it must parse vars before invoking the command, and thus vars are interpolated into the text, which the command is. If the shell does the sane thing of interpreting the command before evaluating vars, it means breaking off from the normal semantics and thus being inconsistent. I imagine the introduction of `[[` and `((` in Bash may have come partly from this reasoning (to abandon the POSIX semantics for these commands), though I never delved far enough into nerdery to learn the difference from `[`—so don't know if they do evaluate vars in the sensible way. I keep wondering if there are any shells or other software calling into utils, that manage to walk the edge of writing commands without bumping into such syntax problems—but avoid having to "enquote" "each" "argument" all the time.
naniwaduni 6y ago
[ is not, in principle, the shell's own function. It merely happens to be implemented as one in most shells anyone actually uses as an optimization.

mikedilger 6y ago

I still use this hack without thinking about it in case the variable on the left is empty. But I suppose I should actually be quoting. [ $X == "hello" ] When X is empty, it gives: example.sh: line 5: [: ==: unary operator expected So I fix it (apparently wrongly) with: [ x$X == x"hello" ] I'll make a point of doing this instead: [ "$X" == "hello" ]

koala_man 6y ago
Definitely quote. `[ x$X == x"hello" ]` may fail for a variety of different values of X like `*`, `Rocky_[1976].jpg`, `hello world` and `foo = xfoo -o bar` Quoting works in all those cases.
- naniwaduni 6y ago
  Of the listed, it should only actually fail on the ones with spaces in them (or rather, the ones that get split according to IFS).
  - koala_man 6y ago
    Whether or not it should, it may fail for any kind of glob. This could be either because the glob ends up unintentionally matching something: $ var='*'; touch xfoo xbar $ [ x$var == x"hello" ] bash: [: too many arguments $ var="Rocky_[1976].jpg"; touch xRocky_7.jpg $ [ x$var == x"Rocky_[1976].jpg" ] (false, should be true) or because you have something like nullglob or failglob enabled: $ var="Rocky_[1976].jpg"; shopt -s nullglob $ [ x$var == x"hello" ] bash: [: ==: unary operator expected

wruza 6y ago

To me the question always was what’s the point of [ba]sh for non-interactive shell scripting, when there is perl (I’m aware that it has issues with versions, portability to non-PCs, etc, so think “theoretical standardized lightweight perl mode for sh”, not a real one). Shell syntax and semantics are a pile of ugly hacks and landmines, which were done because everyone wanted to program cli. When you want to program, just take a programming language, not a poor excuse of it. The amount of time you end up wasting with bash is much greater than reading perldoc *, and mastering bash never returns an investment.

jude- 6y ago

People complaining about how underpowered shells are should have a look at Ion [1]. It's mostly the same as bash, but with first-class support for arrays, maps, byte slices, variable scoping, and (primitive) type-checking. Oh, and it's written in Rust. [1] https://gitlab.redox-os.org/redox-os/ion https://gitlab.redox-os.org/redox-os/ion

svnpenn 6y ago
Last release was 3 years ago...
- jude- 6y ago
  Last commit was 2 weeks ago.
chubot 6y ago
BTW Ion is mentioned here along with many other alternative shells: https://github.com/oilshell/oil/wiki/Alternative-Shells https://github.com/oilshell/oil/wiki/Alternative-Shells

deleted 6y ago

[deleted]