19 ms·
Use long flags when scripting (2013)
- jwmhjwmh 6y agoIf you're trying to stick to the POSIX standard, you have to use short options for standard commands like grep, since the long options are GNU extensions.
- nick238 6y agoI've given up trying to be as pure and standard as possible. I could write all my scripts to use `sh`, but `bash` can make things far easier, and in my docker containers is often required by some other package anyway, so may as well just own it.
- stjohnswarts 6y agoI agree, bash vs sh is night and day. Although these days I mostly use python. If I really really need something I can always shell out in python as well.
- Analemma_ 6y agoUnless you count things like Makefiles, I don't think I've ever written or encountered a "script" that was intended for use on multiple *nix flavors. This strikes me as a YAGNI situation: do it if it comes up, but not before.
- lhoursquentin 6y agoYes most of the time the respecting POSIX to the letter is not needed, but it is of course satisfying knowing that your script can run fine on BSDs and other less common distributions :) Though sometimes you don't need to go that far to break stuff, for instance switching from Fedora to Ubuntu. I've seen many scripts fail on debian derivatives because people think using #!/bin/sh as a shebang is fine since it works on their computer where sh was in fact a symlink to bash. But on debian based distributions /bin/sh is often dash, not bash, and dash is basically the strict POSIX subset + local, all fancy stuff like [[ ]], &>, arrays, ... will fail. Though this is less about long options here and more about general shell scripting.
- kazinator 6y agoYou don't know that your script will run fine anywhere, if you've not actually run it there. But, still, that's no reason to adopt a mindset of actively wrecking the chances of such success. (Which is what passive ignorance amounts to, effectively).
- ori_b 6y agoThis attitude is definitely something that makes my life harder. My OS is usually OpenBSD, and I constantly need to fix other people's scripts. It's not difficult to do, but it is annoying. I don't blame people for ignorance, but it'd be nice if people thought about portability. Non-portable scripts can also bite you on Linux, where Debian, for example, will swap out the shell from bash to dash for performance. but others will not. So, even within the Linux ecosystem, you can end up with scripts that behave differently across distros.
- jfrunyon 6y agoSounds to me like your choice of OS is what makes your life harder...
- ori_b 6y agoIt makes my life easier in most ways. This is one of the costs. And, again, this still bites whenever using different linux distros.
- jjgreen 6y agoIt's really useful to have a *BSD user or two in your userbase complaining about your non-POSIX linuxisms, it improves the portability of your code no end ...
- samtheprogram 6y agoPortable shell scripts probably run on your system all the time, e.g. when installing development tools or packages, especially for cross-platform languages like Node.js, Ruby, Python, etc... If a hunk of software wants a shell script, it's gonna be portable. Homebrew comes to mind, and any other piece of software that does the installation via a shell script.
- kazinator 6y agoIf you've ever run a ./configure script before building a program, you have encountered a script intended for use on multiple Unix flavors.
- kelnos 6y agoI think that's a reasonable attitude to have for your own personal scripts that you use on your own machines. But for stuff you do for work it might make sense to go for portability. Where I work most people develop on macOS, and our infra runs on Linux. I happen to develop on Linux, but if I'm writing development scripts that are intended to be run locally, I can't assume that GNU tools will be installed everywhere.
- viraptor 6y agoThere's a good chance you did. Lots of opensource developers use MacOS, so there's at least compatibility between that and Linux.
- dathinab 6y agoDoes anyone care about the standard? Let's be honest it's outdated, has a bad UX and is with the standards I would hold such a standard against today generally not so good. Sure systems seen try to somewhat be POSIX compliance but from my experience not because they care about POSIX but because that happen to overlap with the idea to be somewhat compiland with other similar systems so that porting software/scripts is easier. So if the systems you use support long options for POSIX commands go ahead and use them. Furthermore if the command you can are not in the standard anyway you again can use long options because they are as much standard as the short ones. Let's be honest this leaves very little use-cases where long options are a problem.
- kazinator 6y agoLast published in 2018 (and actively being worked on) is not outdated. https://pubs.opengroup.org/onlinepubs/9699919799/ https://pubs.opengroup.org/onlinepubs/9699919799/ > Sure systems seen try to somewhat be POSIX compliance The mainstream systems you know and use adhere very rigorously to POSIX. GNU Libc, Kernel, Coreutils, ... and their counterparts in BSD Unixes, proprietary Unixes, Cygwin and whatnot all take POSIX seriously. POSIX is very helpful, and smart coders and sysadmins use it as one of their references for required behavior.
- chubot 6y agoLarry Wall said It's easier to port a shell than a shell script in 1998 ... 22 years later I still think that holds. https://news.ycombinator.com/item?id=10104203 https://news.ycombinator.com/item?id=10104203 Although I would agree that there's a slight hole there: you still need to port stuff like coreutils and all the dependencies, which has been done of course. But I'd be happier if something like busybox was actually portable.
- kazinator 6y agoI think it's easier to write a portable shell script today than 30 years ago. The Autoconf system is predicated on the idea that writing a portable shells script is hard, and so we hide the shell programming behind a mountain of M4 macros. It's not necessarily easier to port a shell script today than 22 years ago, which was not written with portability in mind. There are more shells with more extensions, and then beyond the language concerns, and the environments have exploded. A shell script can easily depend on all sorts of cruft you've never heard of. Oh, just install these five things from the following github repos ...
- efrecon 6y agoI usually test my POSIX scripts in an Alpine docker container. It has busybox by default, which only have a few flags and mostly short ones.
- m463 6y agoIn scripts, I usually do stuff like this command \ --longopt1 \ --longopt2 arg \ argument or command | \ command 2 | \ command 3 \ --opt1 \ --opt2 \ arg
- 52-6F-62 6y agoWow. I just realized I'm terrible for this. I usually format my programming pretty verbosely to make it future-readable but I'm terrible for not doing that in my scripts. I'll add a comment or something explaining it but now I feel guilty and should probably go back and rewrite a few lines... I mean—for long scripts I'll break them up but I haven't paid much mind to arguments/flags.
- dmurray 6y agoI do this if I have to look up the right flag to use. If it's something commonly seen like "grep -o" or "tar -xvzf" I assume whoever is maintaining the script (probably future me) will be more familiar with the abbreviations than the long versions.
- 52-6F-62 6y agoYeah good point. It did strike me, though, because I'd just written several scripts running JMeter commands and sort of breezed over how they're written even though I had to go and look up all the flags.
- viseztrance 6y agoIf you have a very long command you can run ctrl x e (hold control press x, than e) and edit it in your $EDITOR.
- InvaderFizz 6y agoI wish I had known this 20 years ago. Thanks!
- inshadows 6y ago
- moritonal 6y agoWhen writing scripts (CI especially) you'll rarely use, please always use long-flags. It'll save so much headache for the next dev who isn't as up to date with the latest short-hand for az-cli or whatever. When live scripting, feel free to use short.
- efrecon 6y agoI think you are right, for all "exotic" tools, long flags are much better and future proof. For regular tools, short flags are so common that they are probably ok.
- watt 6y agoYour regular tool is not my regular tool. Hell, maybe my regular tool today is not going to be all that regular in 3 years.
- stjohnswarts 6y agoHe's talking about thing like sed/awk/grep/find/xargs/tar not new_hotness_util_2020
- efrecon 6y agoI was. Thank you.
- inshadows 6y agoNo, I'm not going to waste keystrokes and increase noise for lazy people. Convince physicists to write force_gravitational instead of Fg, or functional programmers to give up concepts from abstract algebra, and then I'll reconsider. Until then, RTFM just like you do in other fields. curl is so common you should already know what -s means and if not, well, RTFM.
- lucasmullens 6y agoDiscussion from 2013: https://news.ycombinator.com/item?id=5164354 https://news.ycombinator.com/item?id=5164354
- kazinator 6y agoDon't use long flags when scripting, if they have short equivalents, and POSIX only specifies the short equivalents. Even if the stuff will only ever run on one system, the POSIX flags (1) come from a smaller set of options, since POSIX is fairly conservative in its content this area and (2) are generally well known (often three decades old or older). Don't make me read a man page to confirm that "grep --fixed-strings" really is the same thing as the POSIX-standard "grep -F", and not something subtly different.
- maxioatic 6y agoWhy not just "man grep | grep -- --fixed-strings" real quick?
- kazinator 6y agoThese real quick moments add up to wasted time real quick.
- khalilravanna 6y agoWasted time for someone who has all of these memorized. How about all the future wasted time of people having to look up the flags cause they’re not explicitly spelled out? I’m in favor of explicitness because it saves time for future people and gatekeeps less.
- kazinator 6y ago-F is explicitly spelled out. Proof: I can see it. Explicit is the opposite of implicit, and implicit doesn't mean "abbreviated to a single character that is plainly visible". You don't necessarily know what a long option means without looking it up, just because it is long. Firstly, if your native language isn't English, you may have to look it up in a dictionary. The ordinary meanings you find there may not reveal what that word means in the context of the program. So, at that point you're off to the documentation anyway. I know what "hard" and "soft" are. Therefore, is it obvious what "git reset --soft" means? Hardly. Once you know what it means, "--soft" jogs your memory by association a lot better than some "-X". So that would seem like it is less cognitive work. But when we have long options, it encourages the option vocabulary of a program to keep growing, which adds to the cognitive load.
- adrianmonk 6y agoGreat suggestion, but I'd go for a middle ground: use long flags for the long tail, but short flags are fine for stuff that's very commonly used. I think I'll continue to write these: grep -i rm -rf ln -s gzip -v sed -e instead of: grep --ignore-case rm --recursive --force ln --symbolic gzip --verbose sed --expression If you're working on shell scripts, you probably know certain short flags well enough that long flags just add clutter and don't improve readability.
- gkop 6y agoLong flags aren’t just for readability, they also add entropy which contributes defensiveness against typos.
- MaxBarraclough 6y agoRelated to this, they're less likely to do something unexpected. Many programs use -v as a short flag for --version, but some (such as curl) use it as short for --verbose Probably something you'd catch pretty quickly, but still.
- dpcx 6y agoAs does Python, which drives me crazy. -V is version on Python.
- MaxBarraclough 6y agoVery annoying. Java HotSpot uses -version, rather than --version, which is even worse. Don't mess with the standard long-form flag!
- hackerm0nkey 6y agoI get it wrong every damn time. 16 years and counting.
- makecheck 6y agoIt’s not just “easier for a human” but more stable over time as commands evolve, and less likely to do weird things across Unix variants. Some command-line parsers will auto-complete partial options so you’re less likely to see ambiguity errors in future versions if you picked long-form options. (This isn’t completely foolproof, e.g. a command could have "--foo" and later add "--foobar" but it does help in most cases.) And unfortunately, some tools with the same name will use the same letter to mean different things across Unix variants. You are asking for trouble if you aren’t being clear about what you want.
- zbuf 6y ago> Some command-line parsers will auto-complete partial options This sort of behaviour is tortuous, and should be against international laws. Where _adding_ new, unrelated, options to the interface now changes or breaks the behaviour of existing calling scripts. Plus you never know which abbreviations are in use in the wild. It makes it virtually impossible to maintain a stable interface without just freezing it long-hand. I found this in Perl code; I believe it might be the default behaviour in the standard parser? Our developer really did like the philosophy that "the user may want 50 different ways to express the same thing".
- ucarion 6y agoUnfortunately, this isn't quite true. At least if by Unix you mean POSIX, i.e. you include macOS in your considerations. 1. Sadly, short options are in fact more portable than long options if you're targeting POSIX. For instance, macOS ships with versions of ls, rm, et cetera that only support short flags. This is because long options don't exist at all in POSIX (they're formalized here: https://pubs.opengroup.org/onlinepubs/009695399/functions/getopt.html https://pubs.opengroup.org/onlinepubs/009695399/functions/ge...). 2. Auto-completing partial options only happens for long flags, and it's a bit more than "some" parsers; the canonical implementation of "long" options, GNU's getopt_long, has it as a documented feature: > Long option names may be abbreviated if the abbreviation is unique or is an exact match for some defined option. https://linux.die.net/man/3/getopt_long https://linux.die.net/man/3/getopt_long I 100% agree it's a poor feature. You basically entirely preclude yourself from adding features in a backwards-compatible way.
- deleted 6y ago[deleted]
- alquemist 6y ago--r<TAB>ecursive
- jfrunyon 6y ago`rm -rf` is much more recognizable than `rm --recursive --force`. `tar -cvf` is much more recognizable than `tar --create --verbose --file`. Moreover, why are you writing a shell script if it's NOT quick and dirty?
- Saser 6y agoMakefiles are one situation where one is essentially writing small shell scripts, but which are not really supposed to be quick and dirty in my opinion.
- jolux 6y agoI wanted to say that Microsoft explicitly says to use the full argument label and not omit them when writing PowerShell scripts but I can't find the doc.
- annoyingnoob 6y agoI was thinking that the author would really like PowerShell. PowerShell is the most verbose thing I've ever used. I don't really like to type that much for simple things. I say just learn the flags or look them up, it shouldn't be hard or time consuming. When reading scripts, many times you can infer what the flags mean by understanding the inputs and desired outputs.
- jolux 6y agoSometimes, but well-written PowerShell scripts can be surprisingly readable, particularly in comparison to bash.
- entire-name 6y agoAs the saying goes, code is typically written once and read thousands of times. It takes you less than a second to write the full flag name (add a few seconds if you need to look it up), but it will likely save at least a few hundred readings the time to look up the flag if they do not already know it. In addition, full flag names contributes "self-documentation" in many cases, and also potentially makes searching easier in certain cases.
- jfrunyon 6y agoYou're assuming the long version is more well known than the short version. That's a wildly and provably false assumption. Do you know, without referring to the manpage, exactly what `cp --no-dereference --preserve=links --recursive --preserve=all` does? I don't. I can take some guesses based on the names of the options, but those guesses could very easily miss an important corner case. Do you know what `cp -a` does? I do.
- jdmichal 6y agoThis is how I write PowerShell scripts. I use full cmdlet names, full argument switch names, and I even specify argument switches to positional arguments. I also do my best to honor common flags like `-WhatIf`, `-Verbose`, and `-ErrorAction`. So I end up with scripts like this: if (-not $(Test-Path -Path "$Path" -PathType Container)) { Write-Error -Message "Path ($Path) does not exist or is not a directory." -Category InvalidArgument; return; } When you do this properly, it feels like magic. For example, I wrote a script that does local Maven and Docker builds for a bunch of related projects. So I wrote two functions `Build-Maven` and `Build-Docker` with proper common flag support and error handling. Then, when I use them, I just do something like this: $PSDefaultParameterValues = @{ 'Build-Maven:ErrorAction' = 'Stop'; 'Build-Docker:ErrorAction' = 'Stop'; }; Build-Maven "$Path\A"; Build-Maven "$Path\B"; Build-Docker -Path "$Path\B" ` -Dockerfile "$Path\B\Dockerfile" ` -Tag "B:$Tag"; That first clause automatically amends `Build-Maven` and `Build-Docker` commands with `-ErrorAction Stop`. So if any of those build commands fail, the entire script halts there. And, if I pass in `-Verbose` to this script, that's forwarded to the build commands and I'll see the Maven and Docker build output.
- Arnavion 6y agoPS has another reason to use full names of parameters in long-lived scripts. If you use short names like `-Ver` expecting it to match `-Verbose`, a future version of the commandlet can add `-Version` and now `-Ver` is ambiguous. On the other hand, I've seen some people eschew `%` and `?` in favor of writing `ForEach-Object` and `Where-Object`, which in my opinion is too extreme in the other direction. BTW, in: if (-not $(Test-Path -Path "$Path" -PathType Container)) { ... you don't need the `$`, and if $Path is already a string you don't need the `""` either.
- ziml77 6y agoTo me this isn't just another reason, it's THE reason to not use the short form in scripts. Although unlikely, the worst case of this could be pretty nasty. It's possible for someone to remove a flag that was deemed useless and add another flag that matches the same prefix and has a very different effect. I don't know how to feel about % and ?. The thought with not using them is that they are just aliases and could be changed. But that reasoning sort of breaks down since any command could be aliased to something dumb like `Set-Alias Get-Content Remove-Item`.
- Someone 6y agoAre there editors that use autocomplete to suggest arguments? That would make finding and typing the long arguments easier. As it is, it’s easier to type the long arguments in a shell than in an editor.
- bondolo 6y agoI have been following this advice for about five years with my scripts and the best result has been compliments from co-workers and others who have picked up the scripts and felt like they had a much better understanding of what the scripts were doing. Combined with use of shellcheck and shfmt working in shell scripting has come a long way in the last couple years. I still feel like BASH is a dead end--the quoting and whitespace issues combined with the bifurcation caused by Apple not shipping Bash 4.0+ make moving to Python probably the better course in 2020.
- ramses0 6y agoIt's so unfortunate about Bash + Apple. Bash probably deserves to die out, but Apple has really pulled a bait-and-switch on OSX's *nix/bsd underpinnings over the years.
- saagarjha 6y agoDie out and be replaced by what?
- stjohnswarts 6y agoPowershell? It's available for linux now
- nobody9999 6y ago> Powershell? It's available for linux now I'd rather have my tonsils extracted through my ears. I went the other way[0] decades ago and never looked back. [0] https://www.cygwin.com https://www.cygwin.com
- stjohnswarts 6y agoI was mostly joking. The verbosity of powershell just makes me throw up in my mouth a little time every time I have to deal with it.
- noisem4ker 6y agoEnough idiotic Unix crypticism. Time to grow up and be responsible for making the code readable for the next unfortunate person having to mess with *sh scripts. No excuses, no shortcuts, no turning the time sunk learning random letter combinations into entry barriers for newcomers.
- kbr2000 6y agoThe real shortcut here is the next person wanting to google every answer (instead of taking the time to read part of a manpage).
- mehrdadn 6y agoGreat. Now try typing 'touch --no-create foo' on OS X and let me know how it goes.
- ddevault 6y agoAbsolutely not. Long flags are non-standard and non-portable. Don't write GNU scripts - write shell scripts.
- stjohnswarts 6y agoDepends on the situation as always. Obviously if it's going to be used cross platform go for it, otherwise YAGNI
- leipert 6y agoI try to write long form now, but every time I encounter short form and don’t know what it means I just use explainshell: https://explainshell.com/ https://explainshell.com/ Long form also allows me to find things faster in man pages.
- tyingq 6y agoI think adding the -- to the end, to stop option processing is advisable as well. Where that's supported, of course. Some may disagree, but unexpected flags passing through seems dangerous to me.
- mikey_p 6y agoWent to try to update some of my scripts and stuff and found that most of OS X/macOS uses BSD utils which don't have the long flags found in linux versions. For example check out xargs: https://www.freebsd.org/cgi/man.cgi?xargs https://www.freebsd.org/cgi/man.cgi?xargs https://man7.org/linux/man-pages/man1/xargs.1.html https://man7.org/linux/man-pages/man1/xargs.1.html
- atum47 6y agonever thought about that. I usually try to keep my code short to try to impress other programmers, but that's really good advice
- Jowsey 6y agoI'd say a painless, easy to understand program is more impressive than a short one
- kvz 6y agoAgreed, bash3boilerplate.sh has long suggested that a few extra keystrokes in a script pay off in saving your collaborators and future self trips to manpages. Disclosure: b3bp author.
- arendtio 6y agoI wonder a bit, that those POSIX comments are so far in the bottom of this discussion. I mean, you are loosing portability by using long flags (because it is not standard compliant) and yet it seems to be just a footnote in this discussion...
- ainar-g 6y agoNot a lot of people these days care about portability, unfortunately. You'll be lucky if your project's build scripts run the same way on both Debian and vanilla macOS. And when they don't, the answer people will give you is probably something like “Just install GNU tools!”. *BSD? Forget about it.
- hashkb 6y agoIs there not a linter that can autofix this? Seems like it shouldn't be a human responsibility.
- hanselot 6y agoI'm in no way related to the tldr dev/project, but pip install tldr tldr find Will give you a bunch of user submitted examples of common usages of things. I use this most frequently with imagemagick and ffmpeg.
- nojvek 6y agoGreat advice. Except a whole bunch of popular cli utils doesn’t have long versions. Like kubectl -f. In apply, it means file, in logs it means follow. Drive me nuts how they overload short flags with different means but no long version. Or gsutil/gcloud. I get google folks, don’t really care much about always having a long readable version of short flag.
- makz 6y agoNo, thanks