42 ms·

Minimal safe Bash script template

494 points by signa11 6y ago

abiro 6y ago
My rule of thumb: if the script needs argument parsing or any sort of error handling/cleanup, I’ll write it in Python 3.5 instead of bash.
- quyleanh 6y ago
  So true. In my case, if my script has loop, I will use python or perl instead of bash.
  - aduitsis 6y ago
    If the script must iterate over the lines of its input, I've found perl -nlE to be a godsend. Basically it is equivalent to while(<>){ chomp; <insert your -E code here> }
- emptyparadise 6y ago
  Once I type a square bracket in a shell script, I know it's time to bring out Python. I don't think it's humanely possible to write a test command without shooting yourself in the foot in an exciting new way.
  - usr1106 6y ago
    > don't think it's humanely possible to write a test command without shooting yourself in the foot in an exciting new way That's a weird claim. Could it be that you have never really learned shell programming or read the man pages? Or can you give an example? We have lot of production code with many test statements and I don't remember when we were bitten last time by some "surprising feature" of test. I agree that test works differently than comparisions in many programming languages. So you need to understand how it is different. And you need to be disciplined about quoting, otherwise surprises will happen when you have user-controlled input and the shell starts to do word-splitting. But if you use shellcheck you will be reminded of quoting everywhere. It's not a problem specific to test anyway.
    - imtringued 6y ago
      I have written dozens of of bash utilities at a single company. Probably 5000 lines of bash in total. I still don't know the whole syntax for if/for/case and have to look it up every single time.
      - zimpenfish 6y ago
        I've written hundreds (if not thousands) of bash utilities over 20+ years and I still occasionally have to check the syntax for various bits.
    - kristaps 6y ago
      > So you need to understand how it is different. And you need to be disciplined about quoting Or I can use a sane language that let's me get my stuff done faster and with less opportunities to shoot myself in the foot.
    - meowface 6y ago
      If I'm modifying an existing script, I generally have no problem doing that. But the tweet quoted in the article rings very true to me: >The opposite of "it's like riding a bike" is "it's like programming in bash". >A phrase which means that no matter how many times you do something, you will have to re-learn it every single time. I and probably a lot of programmers write bash scripts just enough to remember the vague shape of what we want to do, yet still infrequently enough to require looking up the exact characters every single time (-z? ! -z? one bracket? two brackets? quotes? no quotes? something about a semicolon and "then", right? where can you add newlines, again? oh yeah, it's not "end" like Ruby/Lua but "fi" right? what's "else if", again?). I'm sure if we were working with bash scripts daily, we'd memorize it, but it's just rare and just different enough that it's basically day 1 all over again every time. Shellcheck helps as a safety net, but it doesn't necessarily save me a lot of time and cognitive load. If I'm writing a new script from scratch and I realize there's going to be some branching or prompts or complex argument handling, I just write it in Python. It saves time and effort, it lets other people on my team more quickly modify and debug it, and it's more predictably portable ("oops, only sh, no bash"). And if it ever needs to do something more advanced in the future, it's way easier to extend. Or put much more succinctly, by 'kristaps in another comment: >Or I can use a sane language that let's me get my stuff done faster and with less opportunities to shoot myself in the foot. It's nice for quickly writing something that's just wrapping multiple commands, though.
  - dexen 6y ago
    Awk is a viable replacement for Bash for scripting. A long established standard and installed on pretty much every POSIX system. It has more robust facilities for handling variables, arrays, regex search & substitution, etc.
    - usr1106 6y ago
      awk is a full programming language. But it has gotten little attention in the last 20 years. If you expect that someone else can read and maintain your code with minimal learning I don't think it's the tool of choice.
      - jrochkind1 6y ago
        But... neither is bash. I am sympathetic to the argument that awk is theoretically superior to bash in almost all ways (and it is of course nearly as available as bash). But... something still makes me reluctant to switch to using it in the cases I write a bash script (which I'm already trying to minimize, as is the OP). I'm not really sure why though. (Fun-for-mostly-me fact, my very first programming job ever was writing awk. It was a while ago. I certainly don't remember any awk now).
    - alexhutcheson 6y ago
      Awk is great for snippets that are meant to be copy+pasted into a shell, and is the only reasonable option sometimes if you're stuck in a Busybox environment, but for almost everything else you'd be better off with Python. This previous comment of mine goes into a bit more detail: https://news.ycombinator.com/item?id=23242753 https://news.ycombinator.com/item?id=23242753
    - fomine3 6y ago
      Agreed but I point that not all awks are gawk.
  - vlowther 6y ago
    Or better yet, condition yourself to hit [ a second time and use Bash's built-in conditionals instead of test and [
- usr1106 6y ago
  Depends. In an embedded system I prefer not to have to install Python at all.
  - franga2000 6y ago
    In an embedded system (== very limited resources), wouldn't you want to avoid any interpreted language, even shell? We have some very nice languages that compile to machine code these days
    - im3w1l 6y ago
      BASIC is 56 years old. Computers back then were not very fast, but interpretation was still viable.
      - usr1106 6y ago
        Have used it on 7KB RAM :) But to be fair the interpreter was in ROM, no idea how big that was.
        teddyh 6y ago
        ROM was usually very slow, so all ROM content was usually copied to RAM before execution. RAM size was therefore still an issue.
        usr1106 6y ago
        I am pretty sure the PET-2001 didn't do that. 8 KB was all it had, about 1 KB was used by the operating system and 7 KB where available for programming. Actually something you would call byte code today, every basic command was 1 Byte. Just don't use labels with 4 or 5 characters, that wastes a lot of space! The first models came even with 4 KB RAM / 3 KB usable. Have never seen one of those, though.
    - colonwqbang 6y ago
      If we're talking Linux embedded and not something like an MCU, then you already likely have a shell installed. So you might as well use it for the things which shell excels at. Shell scripts are much smaller in size than the equivalent C code, for those things that shell is typically used for.
- dxdm 6y ago
  Why Python version 3.5 specifically?
  - abiro 6y ago
    I work on Debian or derivatives mostly and the oldest Debian LTS has 3.5.
- loudmax 6y ago
  I agree with the general sentiment, though I set the bar a bit higher (or lower, depending on your perspective). Bash does okay with positional arguments, error handing, and even loops. You can get a lot done with that, and there are a lot of circumstances where a little bash script is just what you need. Where I draw the line is at arrays. If you're starting to think about data structures, bash isn't the right tool for the job.
- dvfjsdhgfv 6y ago
  Well, it depends. I had a co-admin who used to rewrite all my Bash scripts in Python. Most of them were a bunch of os.system() calls. He was a bit upset when I pointed out they are actually buggier than their Bash originals and noticeably slower.
  - Silhouette 6y ago
    Indeed. Shell scripting can easily get overcomplicated and I agree that moving to a real programming language is better at that point. However, Python isn't necessarily the ideal language for it either, because managing child processes with its standard library functionality can get overcomplicated and verbose, particularly once you start dealing with functionality like pipes. So people often resort to generic os.system calls instead and then you get a strange hybrid style.
- fillest 6y ago
  Exactly. Or even stricter. I've finally made a self-contained helper for that: https://gist.github.com/fillest/8d64f8fa0cdb1745bfc9c683cf39d453 https://gist.github.com/fillest/8d64f8fa0cdb1745bfc9c683cf39... Pure Bash is really not needed since we have Python 3.
  - enw 6y ago
    Assuming you can afford introducing the Python dependency. Also pipe/subprocess heavy scripts often become sad complex monstrosities in non-shell languages.
- FriedrichN 6y ago
  Agreed. Not with the programming language per se, but I agree with the general premise. However, in cases where portability is important, bash is would be my choice. If I can pass someone a bash script and not ask them to also install Python and whatever libs I used, it makes my life so much easier.
  - jonfw 6y ago
    Containers work great for me if the target users are also familiar with them
- kps 6y ago
  It's too bad (for bash users) that bash never bothered to embrace ksh getopts. The equivalent of the article's two pages is roughly, usage=$' [-1][+NAME?script - Script synopsis here] [f:flag?Some flag description] [p:param?Some param description]:[string?description] ' while getopts "$usage" opt do typeset opt_${opt}=${OPTARG:-1} done
- ben11kehoe 6y ago
  Agreed about arg parsing. One day I decided to figure out how to get Python arg parsing in a bash script. The horrifying result: https://gist.github.com/benkehoe/c469ebb76ad80d45405077e109352380 https://gist.github.com/benkehoe/c469ebb76ad80d45405077e1093...
  - jaytaylor 6y ago
    Not horrifying, that's pretty cool! Or we are both the same kind of horrifying :D
  - fl0wenol 6y ago
    Dude, that's freaking brilliant. Wanting to steal that one...
  - deleted 6y ago
    [deleted]
rich_sasha 6y ago
It's a very nice template, some great gems in there, like cd "$(dirname "${BASH_SOURCE[0]}")" >/dev/null 2>&1 . My general issue, and it applies here too I guess, is that by the time I have 99 lines of Bash just as boilerplate,, +logic I might as well write it in Python. But if I just have to write Bash, I might use it one day.
- Normal_gaussian 6y ago
  tbh the first three lines are more than enough if you dont need arg parsing or special handling for interrupts. CI and the like where they are (repeated) one shots. Notably the line you highlighted I do as SCRIPT_ROOT="$(cd $(dirname "${BASH_SOURCE[0]}") && pwd)" This preserves the cwd of the invoker (for relative files etc) wholst giving you access to your asset files etc.
- Brian_K_White 6y ago
  That forked process to do something that's built-in for free is one of the worst parts about it, the opposite of a gem.
koolba 6y ago
> The next line does its best to define the script’s location directory, and then we cd to it. Why? Eh? This breaks all relative path arguments: /path/to/foo.sh bar.txt If you need to reference relative path data internally then that should be hidden from the caller by that same logic, but used only internally.
- saurik 6y ago
  Yeah: I almost never want that... if you remarkably want to source files relative to the script location instead of the working directory, I will argue you usually want to do that with a variable for the directory, except in very even-more-special circumstances (in which case I have done something similar to this article).
  - InfiniteRand 6y ago
    Big use case - you want to run a helper script that is located relative to your original script + this is a package of scripts that you move between machines and/or share with co-workers
    - e12e 6y ago
      That is what $PATH is for?
      - rovr138 6y ago
        Assuming it's only 1 set of scripts. I do something similar with scripts that are within projects. There are scripts named the same but vary due to each project being different so I can't use $PATH (unless I export it every single time I change projects and open a terminal for it) Commands are relative and sometimes these can be executed by other things. Other scripts, cron, etc. depending on the environment and the use. So, first step, cd into the correct directory and then run everything relative to it.
        e12e 6y ago
elygre 6y ago
I don't know what this does: set -Eeuo pipefail But I do know this one, which appears to be missing one: set -o pipefail -o errexit -o nounset So, one step up: Use long options and long names. Make it readable, and not just writable.
- phaemon 6y ago
  You're missing `errtrace` (-E) which means "any trap on ERR is inherited by shell functions, command substitutions, and commands executed in a subshell environment" You probably haven't used it much if you haven't been using `trap`
  - usr1106 6y ago
    I use just trap EXIT With set -eu errors in subshells should lead to an exit anyway, so my exit handler is called. Or am I missing some case?
    - phaemon 6y ago
      I can't think of any difference if you're trapping both with the same handler. Probably only if you want a separate handler for ERR vs EXIT.
      - Liskni_si 6y ago
        I often set up a temporary directory at the beginning and then clean it up in an EXIT trap. Adding ERR and errtrace (-E) would remove the tmpdir or the first failure anywhere, no matter if it's a recoverable one. So I'm sticking with just set -eu and trap EXIT...
- usr1106 6y ago
  You are missing: > E If set, any trap on ERR is inherited by shell functions, command substitutions, and commands executed in a subshell environment. The ERR trap is normally not inherited in such cases. set -eu is at least 30 years old. Not sure whether it makes sense to rewrite established code.
graderjs 6y ago
What license is the author releasing it under? I had a look on Maciej's blog and the gist, but couldn't see. I want to save it to possibly use in my projects.
- mradzikowski 6y ago
  Hey, I've added MIT license for all my gists (https://gist.github.com/m-radzikowski/d925ac457478db14c2146deadd0020cd https://gist.github.com/m-radzikowski/d925ac457478db14c2146d...), so feel free to use it.
_false 6y ago
A lot of people here are leaning towards just writing the script in Py. Could somebody please educate me on why bashing it would be preferable?
- azaras 6y ago
  In devops nowadays: Bash is a prototype to the app in Python that is a prototype to the app in Golang.
  - richardwhiuk 6y ago
    which is a prototype to the one in Rust.
    - elitepleb 6y ago
      Which is a prototype to the small C utils that solve the problem better, and bash again
      - drran 6y ago
        Small Rust utils are better than C equivalents. Ripgrep is lot faster than grep, despite slower regex engine.
        kortex 6y ago
        I look forward to the slow but steady reinvention of various coreutils in Rust or Go. Rg-ripgrep is better than grep. Fd-find is better than find. Exa is better than ls/tree. Fzf has no equivalent. I'm still looking for a ping that lets me timeout less than 1 second. There are so many assumptions baked in to the early tools that a complete rewrite just bypasses.
        GoblinSlayer 6y ago
        As far as I see, ripgrep isn't small - a multimegabyte heavily compressed package.
usr1106 6y ago
Not sure how to put it into the template, but do use shellcheck. At least a comment. If you use CI that should run shellcheck to check all scripts.
thomasbacklund 6y ago
If serious about creating shell script "apps", I think standard POSIX shell is the way to go. The Bashism is seldom needed, even though it is powerful. Why? Because shell scripting is often used in DevOps, and containers very often do not come with Bash, but with (busybox) ash or dash instead. Say you are going FROM nginx to FROM nginx:alpine (to get that smaller image size), woha! your bash script is not working anymore.
- CJefferson 6y ago
  My main problem with POSIX shell is it is hard to test, as people might use bash, ash, zsh or dash as sh. Bash is at least not a moving target.
  - saagarjha 6y ago
    You can run it through shellcheck, no?
  - thomasbacklund 6y ago
    Using `shellcheck --shell sh` is a good start :) If adhering to POSIX it should run the same in bash, ash and dash. Bash is kind of a moving target in it self, since it comes with many new features. You have to be aware of which version you are targeting. For instance when using arrays, associative arrays, there is a diff between version 3 and 4. Version 5 is even more competent and it's easy start digging to deep into the cookie jar and getting your hand stuck in there :)
    - patrec 6y ago
      > Using `shellcheck --shell sh` is a good start :) > If adhering to POSIX it should run the same in bash, ash and dash. Not so. Shellcheck will tell you about syntactically invalid bashisms, but will not warn you about the zillions of things that are under or unspecified in posix. Pertinent minimal example: $ echo $'#!/bin/sh\ntrap "echo exiting" EXIT\nsleep 1h' > exit.sh $ shellcheck --shell sh exit.sh && echo "ALL GOOD!" $ /bin/dash exit.sh ^C $ /usr/bin/zsh exit.sh ^C $ /bin/bash exit.sh ^Cexiting Good luck finding a person who can use trap in a way that actually works as intended across different posix conform shells without consulting stack overflow first.
LC_ALL 6y ago
POSIX has multiple Turing complete utilities that can be used in tandem. Its not a very fair competition. Many of the primitives involved are simply not primitives of higher level languages. Write a python script to remove blank lines from standard input. In awk it takes 5 non-whitespace characters (including 'awk'). Use awk to remove blank lines from your python code when you need to paste it into a terminal. Many python developers have a terrible time from the command line because they do not use the shell to help themselves out.
njkleiner 6y ago
If your argument for using the shell (as opposed to a real programming language such as Python) is portability, writing a shell script that is not POSIX compliant literally defeats the point. Also, 100 lines of UX-related bloat hardly classifies as "minimal".
- deleted 6y ago
  [deleted]
- gtsop 6y ago
  > writing a shell script that is not POSIX compliant literally defeats the point. Your argument is invalid. Even if the script isn't posix compliant you are comparingb portability with regards to the availability of bash vs python binaries on your target machine. Do you have data to support python is more likely to exist in linux machines than bash?
  - jedimastert 6y ago
    I think the argument made here is that choosing bash over python because of portability doesn't square with using non-POSIX-compliant bash commands.
  - deleted 6y ago
    [deleted]
  - njkleiner 6y ago
    > Do you have data to support python is more likely to exist in linux machines than bash? That's not what I'm saying. I never claimed Python was more widely supported that Bash, in terms of this argument you should use neither, that's the whole point. What I'm saying is that it does not make sense to use Bash over e.g. Python in favor of supporting a wider range of machines while at the same time limiting support to Bash environments when you could use POSIX instead which is a subset of Bash and therefore more widely supported than Bash and Python. Do you have data to prove that POSIX shell less likely to be supported than Bash and Python respectively? Otherwise I don't see how my argument is invalid. Of course Bash support is larger than Python support (that's what you seem to be saying), and therefore even a Bash-specific script is more portable. What I'm suggesting is that it's contradictory to stop there when you could simply drop the bashisms and go for full POSIX compliance for even more portability. That said, if I'm misunderstanding you, please point out how so.
louwrentius 6y ago
I have a lot of experience writing Bash scripts and I regret It. Now that I have decent enough mastery of python to be dangerous I realize that you will get to a point where you wish you had access to all the niceties of a decent programming language. Any shell script over 50 lines is suspect.
- enriquto 6y ago
  > Now that I have decent enough mastery of python to be dangerous I realize that you will get to a point where you wish you had access to all the niceties of a decent programming language. What decent language is it, then? Or are you implying that it doesn't exist?
  - louwrentius 6y ago
    I was unclear: get to a point in a bash script where I wished I started out in Python.
traceroute66 6y ago
I think its almost hilarious that something calling itself a "safe script template" doesn't even spend a couple of lines dealing with the possibility of race conditions (e.g. a locking process so that if, as is almost inevitable, my script gets put in a cronjob, it doesn't run when already running). The four lines the script wastes on setup_colors() - something that has nothing to do with "safety" - could easily be used for a basic locking process.
- michaelcampbell 6y ago
  While I find the "almost hilarious" kind of overly snarky, I have my own version of this that has the race condition check. Basically uses `mkdir $THISBIN.lock` line that gives me an atomic way to create or check for existence of a logical semaphore, and provides me with a scratch area. The trap'd cleanup function removes it (which when debugging, and I use that scratch area, is inconvenient, but <shrug>)
  - twic 6y ago
    Read to the end before screaming. I do this: exec >>$log_file 2>&1 if ! flock --exclusive --nonblock 1 then log "unable to get exclusive lock, aborting" exit 1 fi Which redirects standard output to a log file, then tries to get an exclusive lock on it. The lock is created by the flock subprocess, but continues to exist until the script exits and closes that file descriptor. This means there can only be one process writing to a given log file at a time. For me, that is the right granularity of locking - i have several cron jobs running this script, but with different arguments, and writing to different log files.
    - AdamGibbins 6y ago
      flock isn't all platforms unfortunately, doesn't exist on MacOS.
      - twic 6y ago
        Quite true. On OS X, this script will do the right thing and not run until you get a proper computer.
binwiederhier 6y ago
There is a lot of "better write this in python"-ism here, and while I agree with the general sentiment that a more complex script shouldn't be written in bash, there are a ton of use cases where it is preferable to have it written in bash. So thank you for this post. It was very educational.
dvfjsdhgfv 6y ago
This website is an excellent example why I browse the web with JS off by default these days. When I turned it on, an obnoxious icon appeared, refusing to go away. When I hovered on it, another box appeared with an x in the corner. I moved the mouse to close it, and the box disappeared. The obnoxious icon, however, stays as before. I'm really glad we are still able to turn JavaScript off these days. I'm pretty sure in the next years more and more so-called "developers" switch to text rendering by JS, rendering more websites unusable.
- petters 6y ago
  Why did you turn it on? For testing?
  - dvfjsdhgfv 6y ago
    Yes, I sometimes turn it on to see what I "lose" by having it switched off by default.
- sim_card_map 6y ago
  > I'm pretty sure in the next years more and more so-called "developers" switch to text rendering by JS, This has been done for years on lots of sites.
  - dvfjsdhgfv 6y ago
    Of course, but it hasn't become the norm yet. I'm afraid this day will eventually arrive as most user-hostile techniques depends on JS being on, so they need to find a way to force it on users.
technimad 6y ago
This script is too much boilerplate and opiniated for me. I maintain a snippet with the most important settings for safe and robust shell script execution: https://gitlab.com/-/snippets/1968838 https://gitlab.com/-/snippets/1968838 I'm now thinking of adding the trap cleanup boilerplate to my snippet as well, as it tends to get overlooked too often.
fn1 6y ago
Really minimal template, that works well for all scenarios and doesn't cause unexpected behaviour (like changing IFS does): #!/usr/bin/env bash set -eEu -o pipefail DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" reference all relative things with dir first: "$DIR/relativestuff.sh"
- nerdponx 6y ago
  Every time I see this I can't help but smile, knowing that in Zsh the equivalent code is: #!/usr/bin/env zsh setopt err_exit no_unset DIR=${0:h}
  - lukeschlather 6y ago
    Honestly I think I prefer the bash version to `DIR=${0:h}`. Although Powershell is superior with simply "$PSScriptRoot" (On the other hand, Powershell's strict mode is a mess and "catch errors" in Powershell is still an extra setting that doesn't actually catch errors half the time.)
    - rhencke 6y ago
      If you like that, you'll love working with PowerShell's exit codes. [1] [2] [3] [1] https://weblogs.asp.net/soever/returning-an-exit-code-from-a-powershell-script https://weblogs.asp.net/soever/returning-an-exit-code-from-a... [2] https://octopus.com/blog/powershell-exit-codes https://octopus.com/blog/powershell-exit-codes [3] https://community.idera.com/database-tools/powershell/powertips/b/tips/posts/correctly-returning-exit-codes https://community.idera.com/database-tools/powershell/powert...
    - nerdponx 6y ago
      Out of curiosity, what do you like better about the Bash version?
      - kwhitefoot 6y ago
        Not the OP. For the occasional user, like me, it is more obvious what it is doing.
z29LiTp5qUC30n 6y ago
It should be pushd "$(dirname "${BASH_SOURCE[0]}")" >/dev/null 2>&1 not cd "$(dirname "${BASH_SOURCE[0]}")" >/dev/null 2>&1 with a matching popd at the end Otherwise it can screw up parent scripts
- Galanwe 6y ago
  Not unless the script is source'd, which is very unlikely.
- dastx 6y ago
  Really scripts should avoid called `cd` or `pushd`/`popd` wherever possible. It's a lot better to just work with absolute directories, i.e. as someone else pointed out: DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
  - Brian_K_White 6y ago
    no no no... god this is gross and awful SELF=${0##*/} # aka basename DIR=${0%/*} # aka dirname
    - dastx 6y ago
      Running a script as sh script.sh returns only script.sh, thus, you've already got SELF, but you wouldn't get DIR.
      - cat199 6y ago
        but, you don't need it since you are already there, and can just use $PWD (no need for `pwd`) from that point onward.
arpa 6y ago
this is not minimal, it belongs in a bash framework.
nicwolff 6y ago
Bash has let you indent heredocs with `<<-` since version 4!
- ben509 6y ago
  Everyone is stuck on the 3.2 BSD version of bash. T_T
tutfbhuf 6y ago
> The downside would be to have to always attach this second file everywhere, losing the “simple Bash script” idea along the way. Put it on GitHub and curl it from there with checksum. Cache the file somewhere and lookup cache first.
rustshellscript 6y ago
I have developed a similar library before: https://github.com/rust-shell-script/bash_cmd_lib/blob/master/src/cmd_lib.sh https://github.com/rust-shell-script/bash_cmd_lib/blob/maste... which can do temp files cleanup, different levels ofreporting and redirecting all error msgs into log file automatically.
jancsika 6y ago
> Just like JavaScript, it won’t go away easily. No, not "just like." Whatever you think of JS the tweet above this quote doesn't apply-- I don't constantly have to look up how a function callback works, or how to do math. To be "just like" shell scripting, you'd have to get JS devs to start regularly using `eval` in lots of unnecessary places, plus using arbitrary, inconsistent use of single-letter aliases for some of the core methods I mean... > set -Eeuo pipefail Upper and lowercase! And the flag meanings aren't even explained in this article! Edit: To be honest, this reflects my own mental model for shell commands: commandName -garblegooklegarblegookle myArg1 myArg2 etc..
- deleted 6y ago
  [deleted]
- rhencke 6y ago
  $ help set ... -e Exit immediately if a command exits with a non-zero status. -E If set, the ERR trap is inherited by shell functions. -u Treat unset variables as an error when substituting. -o option-name Set the variable corresponding to option-name: pipefail the return value of a pipeline is the status of the last command to exit with a non-zero status, or zero if no command exited with a non-zero status
- mradzikowski 6y ago
  Hey, the flag meanings are explained in the article I linked in the post: > For details on what options exactly set -Eeuo pipefail changes and how they will protect you, I refer you to the article I have in my bookmarks for a few years now. Link: https://vaneyckt.io/posts/safer_bash_scripts_with_set_euxo_pipefail/ https://vaneyckt.io/posts/safer_bash_scripts_with_set_euxo_p... About "just like JavaScript" - it was sarcasm ;-) I write in TypeScript daily and still make fun of JS. But I agree, Bash is worse!
mradzikowski 6y ago
I'm this blog post (and script) author. I got some opinions and code review feedback here, on Reddit, and blog itself. And I appreciate it! I updated the script template applying the most commonly mentioned things, like removing "cd" to the directory. See details in the article.
- cratermoon 6y ago
  This reads like perhaps a bad copy/paste garbled the sentence: "Just remember that the cleanup() can be called not only at the end but as well having the script done any part of the work. "
yarinr 6y ago
> The opposite of "it's like riding a bike" is "it's like programming in bash". > A phrase which means that no matter how many times you do something, you will have to re-learn it every single time. Couldn't phrase it better myself. After wasting way too much time, I tend to go with Python for anything more than a couple of lines, even trivial stuff. That's, of course, unless there's a very compelling reason to use bash.
- hedora 6y ago
  I’m the opposite. Every time a python script throws an exception at me (pretty much every time I invoke a python script for the first time), my first debugging step is to read it and attempt to rewrite it in bash. 9 times out of 10, the result is 10-100x times shorter, with zero external dependencies, and is also easier to debug. In the other case, the python thing is some non-trivial monstrosity that should have been implemented in a compiled language with static type checking.
  - busterarm 6y ago
    Give me $1 for every time I see a whole python script with loads of dependencies for the simplest sed/awk/jq task with pipes.
    - rmrfstar 6y ago
      Python is readable, awk is not. I'd rather spend an extra 5 minutes writing something that I don't have to burn 20 minutes understanding in just a few months time. I think the popularity of Python for scripting is pretty good evidence that PG was dead wrong on the importance of brevity in programming languages. Further evidence can be found in Python's development of type hints.
      - sillysaurusx 6y ago
        Python isn't that readable when piping. From https://docs.python.org/3/library/subprocess.html#replacing-shell-pipeline https://docs.python.org/3/library/subprocess.html#replacing-... output=$(dmesg | grep hda) becomes p1 = Popen(["dmesg"], stdout=PIPE) p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE) p1.stdout.close() # Allow p1 to receive a SIGPIPE if p2 exits. output = p2.communicate()[0] This isn't a contrived example. One reason I like using macros in Python is because it simplifies exactly this boilerplate. You can write a simplifying function, but programs rarely do.
mongol 6y ago
It does not look minimal. For example, strip the colors
a_zaydak 6y ago
Thanks for the solid article! Reminds me about some bash writing that I have been avoiding.
ddevault 6y ago
Please consider writing your scripts in POSIX sh instead of bash. The spec is here, it's pretty easy to understand and use: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V... Most of your scripts are probably already 90% POSIX compatible and could bridge the gap with minor modifications.
rhacker 6y ago
Bash is kinda like the last remnants of Perl. I know it's not perl, but it's similar. And it has similar conventions. And like Perl, it gets shit done pretty concisely. It's also a write-only language (easy to write / hard to read). I do find it interesting that bash has kinda become the DevOps root tool. I don't think DevOps would have had it's moment without Bash or if bash was similar to DOS Batch files instead, for example.
- mhd 6y ago
  I agree with the DevOps theory, but precisely for the opposite reason. People wrote all kinds of automated tools etc. to replace weird BOFH shell scripts. Almost no syntactical improvements upon Bourne sh (which was a weird Algol-like in C-land), but introducing plenty of incompatibilities because other systems might have just regular sh or an older version of bash. Compared to Korn Shell or yes, Perl, it's a major step back. (I'd also say it's a lot harder to write than read, given that once you're reading a working script, you've already got most of the whitespace issues and badly formed command argument worked out). The Unix commandline as a way to pipe together commands was a pretty great idea. But every inch you step away from that abstract ideal is just mired in a geological accumulation of oddities and bad design (Plan 9 probably being the closest ever attempt of making that work less painfully)
- qiqitori 6y ago
  TBH, calling Bash 'write-only' makes me a bit sad. Bash should be very easy to read for people familiar with the command line. Except for boilerplate (e.g. set switches, documented in 'help set', but just ignore them if you're just reading) or if switches (documented in 'man test', but you really just need to memorize three or so) and a handful of commands that don't tend to be used outside of scripts -- all have a help or man page. You could argue that DOS batch files are hard to read too. Of course, bash's tacked on features -- regex, array/hash syntax, etc. can get a bit hairy, but it's all highly documented. To me, bash is more the polar opposite: hard* to write, easy to read. * not that hard
Brian_K_White 6y ago
Look... it's not that hard to get the script dir, no cd, no child process... script_dir=${0%/*}
aidenn0 6y ago
I can't stand pipefail, and I'm ambivalent about set -e. It usually does what you want, which is sometimes worse than never doing what you want. It's usually fairly straightforward to test for failure at the toplevel manually with something like "|| die 'useful message'" What I would really like is if set -e would return from a function with an error (and if you wrap your entire function body in a subshell that's how it works). There are still some gotchas though: "false && true" will not cause an error exit, nor will "if someFunction; then ..." exit if someFunction encounters an error, which is a problem if you have e.g.: someFunction() { cd /some/path rm -rf * } And expect "set -e" to prevent the remove command from running in the event of the CD failure. Most shell scripts are short enough that you will notice the missing die in something like foo || die bar baz || die And tools like shellcheck will flag dangerous commands that are missing an error check. So, given a fairly short toplevel, set -e has minimal advantages, and it can't be trusted to work inside a function. Sounds pretty questionable to me.
- graton 6y ago
  Isn't that what set -E is for, when used with set -e? So that an error in the function will also cause the script to terminate. From bash manpage: -E If set, any trap on ERR is inherited by shell functions, command substitutions, and commands executed in a subshell environment. The ERR trap is normally not inherited in such cases.
  - aidenn0 6y ago
    Nope, the -E has to do with the "trap" command. This example happily echoes "hello" to stdout: #!/bin/bash set -euE -o pipefail FOO=/this/does/not/exist foo() { cd "$FOO" echo hello } if foo; then true; fi
    - graton 6y ago
      Well in that case it appears to be because of using "if foo" But that might have been your point in the first place :) I wasn't aware of the change in behavior with an 'if' statement. As a note, trap ERR had the same behavior as 'set -e' from my quick check. This dies: #!/bin/bash set -euE -o pipefail FOO=/this/does/not/exist foo() { cd "$FOO" echo hello } foo echo We do not reach here
wglb 6y ago
So just checking my ~/bin directory tree, and the median line count for my 287 scripts is 4. I am unlikely to add this framework to these. Here is one example called 'date8sl.sh': #!/bin/sh date +%Y/%m/%d This gives the date in a nice slash delimited form, suitable for creating date-structured directories, as in mkdir -pv $(date8sl.sh) The longest of the useful ones is 22 lines. My tendency is to put anything more complicated in code, these days preferring sbcl lisp with :inferior-shell, :cl-readline. Also occasionally python. I'm just uncomfortable with shell scripts long enough to benefit from this framework.
- beefbroccoli 6y ago
  little one liners like that are arguably better implemented as aliases
  - wglb 6y ago
    What advantage do you find with aliases? I like that the shell commands are "one file does one thing."
rgovostes 6y ago
What does this do in a directory containing no .txt files? for f in *.txt; do echo $f done Surprisingly, Bash still executes the body of the loop once, with f containing the literal string "*.txt". This behavior is controlled by the "nullglob" option, which hardly anyone ever sets.
- krebs_liebhaber 6y ago
  I encountered this error a few weeks ago while trying to convert a folder of .webp files to .png. God, I love Bash.
- kevincox 6y ago
  I would argue that this is continent for interactive use. It means I don't have to quote as many things and usually gets a reasonable error if I did intent on matching things. But yes, for scripts you are better served by nullglob.
- qiqitori 6y ago
  See https://mywiki.wooledge.org/BashPitfalls https://mywiki.wooledge.org/BashPitfalls for a large collection of similar edge cases. (Recently discussed on HN: https://news.ycombinator.com/item?id=24401085 https://news.ycombinator.com/item?id=24401085)
gkfasdfasdf 6y ago
This is awesome. Question: [[ -z "${NO_COLOR-}" ]] && [[ "${TERM-}" != "dumb" ]] Is it not safe to leave off the double quotes here, because it is inside [[ ]] ?
- Lt_Riza_Hawkeye 6y ago
  No. I'm not sure what the "-" is doing in "${TERM-}", but if $TERM was for example set to "-z stringhere" then that would be interpreted as two arguments to "[[" instead of one, and would ensure that it always returns false, for example
  - vbernat 6y ago
    The "-" is here to not error when the variable does not exist (as the topmost set is configured to trigger error on unset variables). It is meant to substitute the (empty) string on the right if the left variable does not exist.
- ben509 6y ago
  There's normally no need for ${var-} to get default values, and you can put && inside. I'd write that as: [[ -z $NO_COLOR && $TERM != dumb ]] As another comment pointed out, they're erroring on unset values. I don't think the line noise is worth it, but it will catch spelling mistakes, I guess.
graton 6y ago
For when things go wrong in your bash script, I really like the following: # Enable verbose logging of the bash script PS4='+ ${BASH_SOURCE:-}:${FUNCNAME[0]:-}:L${LINENO:-}: ' set -x This improves the tracing function greatly (at the cost of much higher verbosity) in that you get the filename, function, and line number in the output.
krebs_liebhaber 6y ago
> Published by Maciej Radzikowski This guy was dealt a bad hand by God.
pierrebai 6y ago
Shorter minimal safe script template: python. Seriously, give up on OS-specific, version-nightmare, quoting-hell, hairy-corner-behaviour shell scripts. I used to install Cygwin and Vim on all new Windows computers. I now install Python and VSCode on all OS. My life has never been better.
CalebJohn 6y ago
A lot of people in this thread seem to share my desire to replace some of what bash does with scripts. Which is why I wrote a utility to allow writing scripts in-between pipes [1]. The basic idea is to call this tool anytime you would be setting up a complex chain of unix pipes, and instead do the scripting in a an interactive environment that allows you to see output as you type. It's just a toy implementation right now and only supports python, but could eventually support any scripting environment (even bash). I don't really see this as a serious tool, but it can be useful for the odd shell job. Or just for those who like immediate evaluation on every key stroke ;) [1]: https://github.com/CalebJohn/shell-coupler https://github.com/CalebJohn/shell-coupler
ben509 6y ago
> This line does its best to define the script’s location directory, ~~and then we cd to it. Why?~~ (struck through in original) I've done this to make little bash utilities, and while it's cool when it works, it really doesn't work. Never try to guess the script's location. You're adding very weird, broken, magical behavior with no way to test it. So keep it simple and write these scripts to expect help from the user. If you need resources for a bash script, expect them to be in the working directory, or a standard directory, or have the user pass paths in through environment variables or other means. Yes, it's gross and inelegant. It also won't break mysteriously.
combatentropy 6y ago
Does anyone write their shell scripts in JavaScript, calling them with node?
baby 6y ago
Is there a modern alternative for bash that one can use to write bash script without having to write python? Something that looks decent if you are writing an if else, or for loop, etc.
- charrondev 6y ago
  Nothing that is common and built in that meets these requirements and is very legible AFAIK. I tend to write most of my scripts in PHP so I don’t have much of a leg to stand on other than that it is: - very scriptable. - common in the environments I work in. - can easily shell out. - can be either completely loosely typed or more strongly typed as you wish. - has an extensive standard library.
gurjeet 6y ago
Shameless plug; I now use Golang for all my scripting needs. I wrote about it and shared a sample here: https://news.ycombinator.com/item?id=24392172 https://news.ycombinator.com/item?id=24392172
mjgs 6y ago
I have over the years developed a similar template. It’s got usage, arg parsing, etc. One thing I find incredibly useful, which doesn’t appear in the minimal template presented here (unless I’ve missed it), is the ability to run the program in a mode where it prints out debug info like variables getting set, function names and line numbers. I add this to the start of every bash script: if [ -n "$DEBUG" ]; then echo "$0: Setting bash option -x for debug" PS4='+($(basename ${BASH_SOURCE}):${LINENO}): ${FUNCNAME[0]:+${FUNCNAME[0]}(): }' set -x fi Then to run with debug info printed just set DEBUG=1 Ideally there would be a debugger where you could step through code, but this is the closest I’ve found to that.
lenkite 6y ago
Prefer environment variables instead of command line options for bash scripts. More readable and maintainable and testable.
vietvu 6y ago
I love this template. I created one on my own but lacks parse params. However, I normally use Python script for anything needs more than 2 args.

494 points by signa11 6y ago

My rule of thumb: if the script needs argument parsing or any sort of error handling/cleanup, I’ll write it in Python 3.5 instead of bash.

quyleanh 6y ago
So true. In my case, if my script has loop, I will use python or perl instead of bash.
- aduitsis 6y ago
  If the script must iterate over the lines of its input, I've found perl -nlE to be a godsend. Basically it is equivalent to while(<>){ chomp; <insert your -E code here> }
emptyparadise 6y ago
Once I type a square bracket in a shell script, I know it's time to bring out Python. I don't think it's humanely possible to write a test command without shooting yourself in the foot in an exciting new way.
- usr1106 6y ago
  > don't think it's humanely possible to write a test command without shooting yourself in the foot in an exciting new way That's a weird claim. Could it be that you have never really learned shell programming or read the man pages? Or can you give an example? We have lot of production code with many test statements and I don't remember when we were bitten last time by some "surprising feature" of test. I agree that test works differently than comparisions in many programming languages. So you need to understand how it is different. And you need to be disciplined about quoting, otherwise surprises will happen when you have user-controlled input and the shell starts to do word-splitting. But if you use shellcheck you will be reminded of quoting everywhere. It's not a problem specific to test anyway.
  - imtringued 6y ago
    I have written dozens of of bash utilities at a single company. Probably 5000 lines of bash in total. I still don't know the whole syntax for if/for/case and have to look it up every single time.
    - zimpenfish 6y ago
      I've written hundreds (if not thousands) of bash utilities over 20+ years and I still occasionally have to check the syntax for various bits.
  - kristaps 6y ago
    > So you need to understand how it is different. And you need to be disciplined about quoting Or I can use a sane language that let's me get my stuff done faster and with less opportunities to shoot myself in the foot.
  - meowface 6y ago
    If I'm modifying an existing script, I generally have no problem doing that. But the tweet quoted in the article rings very true to me: >The opposite of "it's like riding a bike" is "it's like programming in bash". >A phrase which means that no matter how many times you do something, you will have to re-learn it every single time. I and probably a lot of programmers write bash scripts just enough to remember the vague shape of what we want to do, yet still infrequently enough to require looking up the exact characters every single time (-z? ! -z? one bracket? two brackets? quotes? no quotes? something about a semicolon and "then", right? where can you add newlines, again? oh yeah, it's not "end" like Ruby/Lua but "fi" right? what's "else if", again?). I'm sure if we were working with bash scripts daily, we'd memorize it, but it's just rare and just different enough that it's basically day 1 all over again every time. Shellcheck helps as a safety net, but it doesn't necessarily save me a lot of time and cognitive load. If I'm writing a new script from scratch and I realize there's going to be some branching or prompts or complex argument handling, I just write it in Python. It saves time and effort, it lets other people on my team more quickly modify and debug it, and it's more predictably portable ("oops, only sh, no bash"). And if it ever needs to do something more advanced in the future, it's way easier to extend. Or put much more succinctly, by 'kristaps in another comment: >Or I can use a sane language that let's me get my stuff done faster and with less opportunities to shoot myself in the foot. It's nice for quickly writing something that's just wrapping multiple commands, though.
- dexen 6y ago
  Awk is a viable replacement for Bash for scripting. A long established standard and installed on pretty much every POSIX system. It has more robust facilities for handling variables, arrays, regex search & substitution, etc.
  - usr1106 6y ago
    awk is a full programming language. But it has gotten little attention in the last 20 years. If you expect that someone else can read and maintain your code with minimal learning I don't think it's the tool of choice.
    - jrochkind1 6y ago
      But... neither is bash. I am sympathetic to the argument that awk is theoretically superior to bash in almost all ways (and it is of course nearly as available as bash). But... something still makes me reluctant to switch to using it in the cases I write a bash script (which I'm already trying to minimize, as is the OP). I'm not really sure why though. (Fun-for-mostly-me fact, my very first programming job ever was writing awk. It was a while ago. I certainly don't remember any awk now).
  - alexhutcheson 6y ago
    Awk is great for snippets that are meant to be copy+pasted into a shell, and is the only reasonable option sometimes if you're stuck in a Busybox environment, but for almost everything else you'd be better off with Python. This previous comment of mine goes into a bit more detail: https://news.ycombinator.com/item?id=23242753 https://news.ycombinator.com/item?id=23242753
  - fomine3 6y ago
    Agreed but I point that not all awks are gawk.
- vlowther 6y ago
  Or better yet, condition yourself to hit [ a second time and use Bash's built-in conditionals instead of test and [
usr1106 6y ago
Depends. In an embedded system I prefer not to have to install Python at all.
- franga2000 6y ago
  In an embedded system (== very limited resources), wouldn't you want to avoid any interpreted language, even shell? We have some very nice languages that compile to machine code these days
  - im3w1l 6y ago
    BASIC is 56 years old. Computers back then were not very fast, but interpretation was still viable.
    - usr1106 6y ago
      Have used it on 7KB RAM :) But to be fair the interpreter was in ROM, no idea how big that was.
      - teddyh 6y ago
        ROM was usually very slow, so all ROM content was usually copied to RAM before execution. RAM size was therefore still an issue.
        usr1106 6y ago
        I am pretty sure the PET-2001 didn't do that. 8 KB was all it had, about 1 KB was used by the operating system and 7 KB where available for programming. Actually something you would call byte code today, every basic command was 1 Byte. Just don't use labels with 4 or 5 characters, that wastes a lot of space! The first models came even with 4 KB RAM / 3 KB usable. Have never seen one of those, though.
  - colonwqbang 6y ago
    If we're talking Linux embedded and not something like an MCU, then you already likely have a shell installed. So you might as well use it for the things which shell excels at. Shell scripts are much smaller in size than the equivalent C code, for those things that shell is typically used for.
dxdm 6y ago
Why Python version 3.5 specifically?
- abiro 6y ago
  I work on Debian or derivatives mostly and the oldest Debian LTS has 3.5.
loudmax 6y ago
I agree with the general sentiment, though I set the bar a bit higher (or lower, depending on your perspective). Bash does okay with positional arguments, error handing, and even loops. You can get a lot done with that, and there are a lot of circumstances where a little bash script is just what you need. Where I draw the line is at arrays. If you're starting to think about data structures, bash isn't the right tool for the job.
dvfjsdhgfv 6y ago
Well, it depends. I had a co-admin who used to rewrite all my Bash scripts in Python. Most of them were a bunch of os.system() calls. He was a bit upset when I pointed out they are actually buggier than their Bash originals and noticeably slower.
- Silhouette 6y ago
  Indeed. Shell scripting can easily get overcomplicated and I agree that moving to a real programming language is better at that point. However, Python isn't necessarily the ideal language for it either, because managing child processes with its standard library functionality can get overcomplicated and verbose, particularly once you start dealing with functionality like pipes. So people often resort to generic os.system calls instead and then you get a strange hybrid style.
fillest 6y ago
Exactly. Or even stricter. I've finally made a self-contained helper for that: https://gist.github.com/fillest/8d64f8fa0cdb1745bfc9c683cf39d453 https://gist.github.com/fillest/8d64f8fa0cdb1745bfc9c683cf39... Pure Bash is really not needed since we have Python 3.
- enw 6y ago
  Assuming you can afford introducing the Python dependency. Also pipe/subprocess heavy scripts often become sad complex monstrosities in non-shell languages.
FriedrichN 6y ago
Agreed. Not with the programming language per se, but I agree with the general premise. However, in cases where portability is important, bash is would be my choice. If I can pass someone a bash script and not ask them to also install Python and whatever libs I used, it makes my life so much easier.
- jonfw 6y ago
  Containers work great for me if the target users are also familiar with them
kps 6y ago
It's too bad (for bash users) that bash never bothered to embrace ksh getopts. The equivalent of the article's two pages is roughly, usage=$' [-1][+NAME?script - Script synopsis here] [f:flag?Some flag description] [p:param?Some param description]:[string?description] ' while getopts "$usage" opt do typeset opt_${opt}=${OPTARG:-1} done
ben11kehoe 6y ago
Agreed about arg parsing. One day I decided to figure out how to get Python arg parsing in a bash script. The horrifying result: https://gist.github.com/benkehoe/c469ebb76ad80d45405077e109352380 https://gist.github.com/benkehoe/c469ebb76ad80d45405077e1093...
- jaytaylor 6y ago
  Not horrifying, that's pretty cool! Or we are both the same kind of horrifying :D
- fl0wenol 6y ago
  Dude, that's freaking brilliant. Wanting to steal that one...
- deleted 6y ago
  [deleted]

rich_sasha 6y ago

It's a very nice template, some great gems in there, like cd "$(dirname "${BASH_SOURCE[0]}")" >/dev/null 2>&1 . My general issue, and it applies here too I guess, is that by the time I have 99 lines of Bash just as boilerplate,, +logic I might as well write it in Python. But if I just have to write Bash, I might use it one day.

Normal_gaussian 6y ago
tbh the first three lines are more than enough if you dont need arg parsing or special handling for interrupts. CI and the like where they are (repeated) one shots. Notably the line you highlighted I do as SCRIPT_ROOT="$(cd $(dirname "${BASH_SOURCE[0]}") && pwd)" This preserves the cwd of the invoker (for relative files etc) wholst giving you access to your asset files etc.
Brian_K_White 6y ago
That forked process to do something that's built-in for free is one of the worst parts about it, the opposite of a gem.

koolba 6y ago

> The next line does its best to define the script’s location directory, and then we cd to it. Why? Eh? This breaks all relative path arguments: /path/to/foo.sh bar.txt If you need to reference relative path data internally then that should be hidden from the caller by that same logic, but used only internally.

saurik 6y ago
Yeah: I almost never want that... if you remarkably want to source files relative to the script location instead of the working directory, I will argue you usually want to do that with a variable for the directory, except in very even-more-special circumstances (in which case I have done something similar to this article).
- InfiniteRand 6y ago
  Big use case - you want to run a helper script that is located relative to your original script + this is a package of scripts that you move between machines and/or share with co-workers
  - e12e 6y ago
    That is what $PATH is for?
    - rovr138 6y ago
      Assuming it's only 1 set of scripts. I do something similar with scripts that are within projects. There are scripts named the same but vary due to each project being different so I can't use $PATH (unless I export it every single time I change projects and open a terminal for it) Commands are relative and sometimes these can be executed by other things. Other scripts, cron, etc. depending on the environment and the use. So, first step, cd into the correct directory and then run everything relative to it.
      - e12e 6y ago

elygre 6y ago

I don't know what this does: set -Eeuo pipefail But I do know this one, which appears to be missing one: set -o pipefail -o errexit -o nounset So, one step up: Use long options and long names. Make it readable, and not just writable.

phaemon 6y ago
You're missing `errtrace` (-E) which means "any trap on ERR is inherited by shell functions, command substitutions, and commands executed in a subshell environment" You probably haven't used it much if you haven't been using `trap`
- usr1106 6y ago
  I use just trap EXIT With set -eu errors in subshells should lead to an exit anyway, so my exit handler is called. Or am I missing some case?
  - phaemon 6y ago
    I can't think of any difference if you're trapping both with the same handler. Probably only if you want a separate handler for ERR vs EXIT.
    - Liskni_si 6y ago
      I often set up a temporary directory at the beginning and then clean it up in an EXIT trap. Adding ERR and errtrace (-E) would remove the tmpdir or the first failure anywhere, no matter if it's a recoverable one. So I'm sticking with just set -eu and trap EXIT...
usr1106 6y ago
You are missing: > E If set, any trap on ERR is inherited by shell functions, command substitutions, and commands executed in a subshell environment. The ERR trap is normally not inherited in such cases. set -eu is at least 30 years old. Not sure whether it makes sense to rewrite established code.

graderjs 6y ago

What license is the author releasing it under? I had a look on Maciej's blog and the gist, but couldn't see. I want to save it to possibly use in my projects.

mradzikowski 6y ago
Hey, I've added MIT license for all my gists (https://gist.github.com/m-radzikowski/d925ac457478db14c2146deadd0020cd https://gist.github.com/m-radzikowski/d925ac457478db14c2146d...), so feel free to use it.

_false 6y ago

A lot of people here are leaning towards just writing the script in Py. Could somebody please educate me on why bashing it would be preferable?

azaras 6y ago
In devops nowadays: Bash is a prototype to the app in Python that is a prototype to the app in Golang.
- richardwhiuk 6y ago
  which is a prototype to the one in Rust.
  - elitepleb 6y ago
    Which is a prototype to the small C utils that solve the problem better, and bash again
    - drran 6y ago
      Small Rust utils are better than C equivalents. Ripgrep is lot faster than grep, despite slower regex engine.
      - kortex 6y ago
        I look forward to the slow but steady reinvention of various coreutils in Rust or Go. Rg-ripgrep is better than grep. Fd-find is better than find. Exa is better than ls/tree. Fzf has no equivalent. I'm still looking for a ping that lets me timeout less than 1 second. There are so many assumptions baked in to the early tools that a complete rewrite just bypasses.
      - GoblinSlayer 6y ago
        As far as I see, ripgrep isn't small - a multimegabyte heavily compressed package.

usr1106 6y ago

Not sure how to put it into the template, but do use shellcheck. At least a comment. If you use CI that should run shellcheck to check all scripts.

thomasbacklund 6y ago

If serious about creating shell script "apps", I think standard POSIX shell is the way to go. The Bashism is seldom needed, even though it is powerful. Why? Because shell scripting is often used in DevOps, and containers very often do not come with Bash, but with (busybox) ash or dash instead. Say you are going FROM nginx to FROM nginx:alpine (to get that smaller image size), woha! your bash script is not working anymore.

CJefferson 6y ago
My main problem with POSIX shell is it is hard to test, as people might use bash, ash, zsh or dash as sh. Bash is at least not a moving target.
- saagarjha 6y ago
  You can run it through shellcheck, no?
- thomasbacklund 6y ago
  Using `shellcheck --shell sh` is a good start :) If adhering to POSIX it should run the same in bash, ash and dash. Bash is kind of a moving target in it self, since it comes with many new features. You have to be aware of which version you are targeting. For instance when using arrays, associative arrays, there is a diff between version 3 and 4. Version 5 is even more competent and it's easy start digging to deep into the cookie jar and getting your hand stuck in there :)
  - patrec 6y ago
    > Using `shellcheck --shell sh` is a good start :) > If adhering to POSIX it should run the same in bash, ash and dash. Not so. Shellcheck will tell you about syntactically invalid bashisms, but will not warn you about the zillions of things that are under or unspecified in posix. Pertinent minimal example: $ echo $'#!/bin/sh\ntrap "echo exiting" EXIT\nsleep 1h' > exit.sh $ shellcheck --shell sh exit.sh && echo "ALL GOOD!" $ /bin/dash exit.sh ^C $ /usr/bin/zsh exit.sh ^C $ /bin/bash exit.sh ^Cexiting Good luck finding a person who can use trap in a way that actually works as intended across different posix conform shells without consulting stack overflow first.

LC_ALL 6y ago

POSIX has multiple Turing complete utilities that can be used in tandem. Its not a very fair competition. Many of the primitives involved are simply not primitives of higher level languages. Write a python script to remove blank lines from standard input. In awk it takes 5 non-whitespace characters (including 'awk'). Use awk to remove blank lines from your python code when you need to paste it into a terminal. Many python developers have a terrible time from the command line because they do not use the shell to help themselves out.

njkleiner 6y ago

If your argument for using the shell (as opposed to a real programming language such as Python) is portability, writing a shell script that is not POSIX compliant literally defeats the point. Also, 100 lines of UX-related bloat hardly classifies as "minimal".

deleted 6y ago
[deleted]
gtsop 6y ago
> writing a shell script that is not POSIX compliant literally defeats the point. Your argument is invalid. Even if the script isn't posix compliant you are comparingb portability with regards to the availability of bash vs python binaries on your target machine. Do you have data to support python is more likely to exist in linux machines than bash?
- jedimastert 6y ago
  I think the argument made here is that choosing bash over python because of portability doesn't square with using non-POSIX-compliant bash commands.
- deleted 6y ago
  [deleted]
- njkleiner 6y ago
  > Do you have data to support python is more likely to exist in linux machines than bash? That's not what I'm saying. I never claimed Python was more widely supported that Bash, in terms of this argument you should use neither, that's the whole point. What I'm saying is that it does not make sense to use Bash over e.g. Python in favor of supporting a wider range of machines while at the same time limiting support to Bash environments when you could use POSIX instead which is a subset of Bash and therefore more widely supported than Bash and Python. Do you have data to prove that POSIX shell less likely to be supported than Bash and Python respectively? Otherwise I don't see how my argument is invalid. Of course Bash support is larger than Python support (that's what you seem to be saying), and therefore even a Bash-specific script is more portable. What I'm suggesting is that it's contradictory to stop there when you could simply drop the bashisms and go for full POSIX compliance for even more portability. That said, if I'm misunderstanding you, please point out how so.

louwrentius 6y ago

I have a lot of experience writing Bash scripts and I regret It. Now that I have decent enough mastery of python to be dangerous I realize that you will get to a point where you wish you had access to all the niceties of a decent programming language. Any shell script over 50 lines is suspect.

enriquto 6y ago
> Now that I have decent enough mastery of python to be dangerous I realize that you will get to a point where you wish you had access to all the niceties of a decent programming language. What decent language is it, then? Or are you implying that it doesn't exist?
- louwrentius 6y ago
  I was unclear: get to a point in a bash script where I wished I started out in Python.

traceroute66 6y ago

I think its almost hilarious that something calling itself a "safe script template" doesn't even spend a couple of lines dealing with the possibility of race conditions (e.g. a locking process so that if, as is almost inevitable, my script gets put in a cronjob, it doesn't run when already running). The four lines the script wastes on setup_colors() - something that has nothing to do with "safety" - could easily be used for a basic locking process.

michaelcampbell 6y ago
While I find the "almost hilarious" kind of overly snarky, I have my own version of this that has the race condition check. Basically uses `mkdir $THISBIN.lock` line that gives me an atomic way to create or check for existence of a logical semaphore, and provides me with a scratch area. The trap'd cleanup function removes it (which when debugging, and I use that scratch area, is inconvenient, but <shrug>)
- twic 6y ago
  Read to the end before screaming. I do this: exec >>$log_file 2>&1 if ! flock --exclusive --nonblock 1 then log "unable to get exclusive lock, aborting" exit 1 fi Which redirects standard output to a log file, then tries to get an exclusive lock on it. The lock is created by the flock subprocess, but continues to exist until the script exits and closes that file descriptor. This means there can only be one process writing to a given log file at a time. For me, that is the right granularity of locking - i have several cron jobs running this script, but with different arguments, and writing to different log files.
  - AdamGibbins 6y ago
    flock isn't all platforms unfortunately, doesn't exist on MacOS.
    - twic 6y ago
      Quite true. On OS X, this script will do the right thing and not run until you get a proper computer.

binwiederhier 6y ago

There is a lot of "better write this in python"-ism here, and while I agree with the general sentiment that a more complex script shouldn't be written in bash, there are a ton of use cases where it is preferable to have it written in bash. So thank you for this post. It was very educational.

dvfjsdhgfv 6y ago

This website is an excellent example why I browse the web with JS off by default these days. When I turned it on, an obnoxious icon appeared, refusing to go away. When I hovered on it, another box appeared with an x in the corner. I moved the mouse to close it, and the box disappeared. The obnoxious icon, however, stays as before. I'm really glad we are still able to turn JavaScript off these days. I'm pretty sure in the next years more and more so-called "developers" switch to text rendering by JS, rendering more websites unusable.

petters 6y ago
Why did you turn it on? For testing?
- dvfjsdhgfv 6y ago
  Yes, I sometimes turn it on to see what I "lose" by having it switched off by default.
sim_card_map 6y ago
> I'm pretty sure in the next years more and more so-called "developers" switch to text rendering by JS, This has been done for years on lots of sites.
- dvfjsdhgfv 6y ago
  Of course, but it hasn't become the norm yet. I'm afraid this day will eventually arrive as most user-hostile techniques depends on JS being on, so they need to find a way to force it on users.

technimad 6y ago

This script is too much boilerplate and opiniated for me. I maintain a snippet with the most important settings for safe and robust shell script execution: https://gitlab.com/-/snippets/1968838 https://gitlab.com/-/snippets/1968838 I'm now thinking of adding the trap cleanup boilerplate to my snippet as well, as it tends to get overlooked too often.

fn1 6y ago

Really minimal template, that works well for all scenarios and doesn't cause unexpected behaviour (like changing IFS does): #!/usr/bin/env bash set -eEu -o pipefail DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" reference all relative things with dir first: "$DIR/relativestuff.sh"

nerdponx 6y ago
Every time I see this I can't help but smile, knowing that in Zsh the equivalent code is: #!/usr/bin/env zsh setopt err_exit no_unset DIR=${0:h}
- lukeschlather 6y ago
  Honestly I think I prefer the bash version to `DIR=${0:h}`. Although Powershell is superior with simply "$PSScriptRoot" (On the other hand, Powershell's strict mode is a mess and "catch errors" in Powershell is still an extra setting that doesn't actually catch errors half the time.)
  - rhencke 6y ago
    If you like that, you'll love working with PowerShell's exit codes. [1] [2] [3] [1] https://weblogs.asp.net/soever/returning-an-exit-code-from-a-powershell-script https://weblogs.asp.net/soever/returning-an-exit-code-from-a... [2] https://octopus.com/blog/powershell-exit-codes https://octopus.com/blog/powershell-exit-codes [3] https://community.idera.com/database-tools/powershell/powertips/b/tips/posts/correctly-returning-exit-codes https://community.idera.com/database-tools/powershell/powert...
  - nerdponx 6y ago
    Out of curiosity, what do you like better about the Bash version?
    - kwhitefoot 6y ago
      Not the OP. For the occasional user, like me, it is more obvious what it is doing.

z29LiTp5qUC30n 6y ago

It should be pushd "$(dirname "${BASH_SOURCE[0]}")" >/dev/null 2>&1 not cd "$(dirname "${BASH_SOURCE[0]}")" >/dev/null 2>&1 with a matching popd at the end Otherwise it can screw up parent scripts

Galanwe 6y ago
Not unless the script is source'd, which is very unlikely.
dastx 6y ago
Really scripts should avoid called `cd` or `pushd`/`popd` wherever possible. It's a lot better to just work with absolute directories, i.e. as someone else pointed out: DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
- Brian_K_White 6y ago
  no no no... god this is gross and awful SELF=${0##*/} # aka basename DIR=${0%/*} # aka dirname
  - dastx 6y ago
    Running a script as sh script.sh returns only script.sh, thus, you've already got SELF, but you wouldn't get DIR.
    - cat199 6y ago
      but, you don't need it since you are already there, and can just use $PWD (no need for `pwd`) from that point onward.

arpa 6y ago

this is not minimal, it belongs in a bash framework.

nicwolff 6y ago

Bash has let you indent heredocs with `<<-` since version 4!

ben509 6y ago
Everyone is stuck on the 3.2 BSD version of bash. T_T

tutfbhuf 6y ago

> The downside would be to have to always attach this second file everywhere, losing the “simple Bash script” idea along the way. Put it on GitHub and curl it from there with checksum. Cache the file somewhere and lookup cache first.

rustshellscript 6y ago

I have developed a similar library before: https://github.com/rust-shell-script/bash_cmd_lib/blob/master/src/cmd_lib.sh https://github.com/rust-shell-script/bash_cmd_lib/blob/maste... which can do temp files cleanup, different levels ofreporting and redirecting all error msgs into log file automatically.

jancsika 6y ago

> Just like JavaScript, it won’t go away easily. No, not "just like." Whatever you think of JS the tweet above this quote doesn't apply-- I don't constantly have to look up how a function callback works, or how to do math. To be "just like" shell scripting, you'd have to get JS devs to start regularly using `eval` in lots of unnecessary places, plus using arbitrary, inconsistent use of single-letter aliases for some of the core methods I mean... > set -Eeuo pipefail Upper and lowercase! And the flag meanings aren't even explained in this article! Edit: To be honest, this reflects my own mental model for shell commands: commandName -garblegooklegarblegookle myArg1 myArg2 etc..

deleted 6y ago
[deleted]
rhencke 6y ago
$ help set ... -e Exit immediately if a command exits with a non-zero status. -E If set, the ERR trap is inherited by shell functions. -u Treat unset variables as an error when substituting. -o option-name Set the variable corresponding to option-name: pipefail the return value of a pipeline is the status of the last command to exit with a non-zero status, or zero if no command exited with a non-zero status
mradzikowski 6y ago
Hey, the flag meanings are explained in the article I linked in the post: > For details on what options exactly set -Eeuo pipefail changes and how they will protect you, I refer you to the article I have in my bookmarks for a few years now. Link: https://vaneyckt.io/posts/safer_bash_scripts_with_set_euxo_pipefail/ https://vaneyckt.io/posts/safer_bash_scripts_with_set_euxo_p... About "just like JavaScript" - it was sarcasm ;-) I write in TypeScript daily and still make fun of JS. But I agree, Bash is worse!

mradzikowski 6y ago

I'm this blog post (and script) author. I got some opinions and code review feedback here, on Reddit, and blog itself. And I appreciate it! I updated the script template applying the most commonly mentioned things, like removing "cd" to the directory. See details in the article.

cratermoon 6y ago
This reads like perhaps a bad copy/paste garbled the sentence: "Just remember that the cleanup() can be called not only at the end but as well having the script done any part of the work. "

yarinr 6y ago

> The opposite of "it's like riding a bike" is "it's like programming in bash". > A phrase which means that no matter how many times you do something, you will have to re-learn it every single time. Couldn't phrase it better myself. After wasting way too much time, I tend to go with Python for anything more than a couple of lines, even trivial stuff. That's, of course, unless there's a very compelling reason to use bash.

hedora 6y ago
I’m the opposite. Every time a python script throws an exception at me (pretty much every time I invoke a python script for the first time), my first debugging step is to read it and attempt to rewrite it in bash. 9 times out of 10, the result is 10-100x times shorter, with zero external dependencies, and is also easier to debug. In the other case, the python thing is some non-trivial monstrosity that should have been implemented in a compiled language with static type checking.
- busterarm 6y ago
  Give me $1 for every time I see a whole python script with loads of dependencies for the simplest sed/awk/jq task with pipes.
  - rmrfstar 6y ago
    Python is readable, awk is not. I'd rather spend an extra 5 minutes writing something that I don't have to burn 20 minutes understanding in just a few months time. I think the popularity of Python for scripting is pretty good evidence that PG was dead wrong on the importance of brevity in programming languages. Further evidence can be found in Python's development of type hints.
    - sillysaurusx 6y ago
      Python isn't that readable when piping. From https://docs.python.org/3/library/subprocess.html#replacing-shell-pipeline https://docs.python.org/3/library/subprocess.html#replacing-... output=$(dmesg | grep hda) becomes p1 = Popen(["dmesg"], stdout=PIPE) p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE) p1.stdout.close() # Allow p1 to receive a SIGPIPE if p2 exits. output = p2.communicate()[0] This isn't a contrived example. One reason I like using macros in Python is because it simplifies exactly this boilerplate. You can write a simplifying function, but programs rarely do.

mongol 6y ago

It does not look minimal. For example, strip the colors

a_zaydak 6y ago

Thanks for the solid article! Reminds me about some bash writing that I have been avoiding.

ddevault 6y ago

Please consider writing your scripts in POSIX sh instead of bash. The spec is here, it's pretty easy to understand and use: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V... Most of your scripts are probably already 90% POSIX compatible and could bridge the gap with minor modifications.

rhacker 6y ago

Bash is kinda like the last remnants of Perl. I know it's not perl, but it's similar. And it has similar conventions. And like Perl, it gets shit done pretty concisely. It's also a write-only language (easy to write / hard to read). I do find it interesting that bash has kinda become the DevOps root tool. I don't think DevOps would have had it's moment without Bash or if bash was similar to DOS Batch files instead, for example.

mhd 6y ago
I agree with the DevOps theory, but precisely for the opposite reason. People wrote all kinds of automated tools etc. to replace weird BOFH shell scripts. Almost no syntactical improvements upon Bourne sh (which was a weird Algol-like in C-land), but introducing plenty of incompatibilities because other systems might have just regular sh or an older version of bash. Compared to Korn Shell or yes, Perl, it's a major step back. (I'd also say it's a lot harder to write than read, given that once you're reading a working script, you've already got most of the whitespace issues and badly formed command argument worked out). The Unix commandline as a way to pipe together commands was a pretty great idea. But every inch you step away from that abstract ideal is just mired in a geological accumulation of oddities and bad design (Plan 9 probably being the closest ever attempt of making that work less painfully)
qiqitori 6y ago
TBH, calling Bash 'write-only' makes me a bit sad. Bash should be very easy to read for people familiar with the command line. Except for boilerplate (e.g. set switches, documented in 'help set', but just ignore them if you're just reading) or if switches (documented in 'man test', but you really just need to memorize three or so) and a handful of commands that don't tend to be used outside of scripts -- all have a help or man page. You could argue that DOS batch files are hard to read too. Of course, bash's tacked on features -- regex, array/hash syntax, etc. can get a bit hairy, but it's all highly documented. To me, bash is more the polar opposite: hard* to write, easy to read. * not that hard

Brian_K_White 6y ago

Look... it's not that hard to get the script dir, no cd, no child process... script_dir=${0%/*}

aidenn0 6y ago

I can't stand pipefail, and I'm ambivalent about set -e. It usually does what you want, which is sometimes worse than never doing what you want. It's usually fairly straightforward to test for failure at the toplevel manually with something like "|| die 'useful message'" What I would really like is if set -e would return from a function with an error (and if you wrap your entire function body in a subshell that's how it works). There are still some gotchas though: "false && true" will not cause an error exit, nor will "if someFunction; then ..." exit if someFunction encounters an error, which is a problem if you have e.g.: someFunction() { cd /some/path rm -rf * } And expect "set -e" to prevent the remove command from running in the event of the CD failure. Most shell scripts are short enough that you will notice the missing die in something like foo || die bar baz || die And tools like shellcheck will flag dangerous commands that are missing an error check. So, given a fairly short toplevel, set -e has minimal advantages, and it can't be trusted to work inside a function. Sounds pretty questionable to me.

graton 6y ago
Isn't that what set -E is for, when used with set -e? So that an error in the function will also cause the script to terminate. From bash manpage: -E If set, any trap on ERR is inherited by shell functions, command substitutions, and commands executed in a subshell environment. The ERR trap is normally not inherited in such cases.
- aidenn0 6y ago
  Nope, the -E has to do with the "trap" command. This example happily echoes "hello" to stdout: #!/bin/bash set -euE -o pipefail FOO=/this/does/not/exist foo() { cd "$FOO" echo hello } if foo; then true; fi
  - graton 6y ago
    Well in that case it appears to be because of using "if foo" But that might have been your point in the first place :) I wasn't aware of the change in behavior with an 'if' statement. As a note, trap ERR had the same behavior as 'set -e' from my quick check. This dies: #!/bin/bash set -euE -o pipefail FOO=/this/does/not/exist foo() { cd "$FOO" echo hello } foo echo We do not reach here

wglb 6y ago

So just checking my ~/bin directory tree, and the median line count for my 287 scripts is 4. I am unlikely to add this framework to these. Here is one example called 'date8sl.sh': #!/bin/sh date +%Y/%m/%d This gives the date in a nice slash delimited form, suitable for creating date-structured directories, as in mkdir -pv $(date8sl.sh) The longest of the useful ones is 22 lines. My tendency is to put anything more complicated in code, these days preferring sbcl lisp with :inferior-shell, :cl-readline. Also occasionally python. I'm just uncomfortable with shell scripts long enough to benefit from this framework.

beefbroccoli 6y ago
little one liners like that are arguably better implemented as aliases
- wglb 6y ago
  What advantage do you find with aliases? I like that the shell commands are "one file does one thing."

rgovostes 6y ago

What does this do in a directory containing no .txt files? for f in *.txt; do echo $f done Surprisingly, Bash still executes the body of the loop once, with f containing the literal string "*.txt". This behavior is controlled by the "nullglob" option, which hardly anyone ever sets.

krebs_liebhaber 6y ago
I encountered this error a few weeks ago while trying to convert a folder of .webp files to .png. God, I love Bash.
kevincox 6y ago
I would argue that this is continent for interactive use. It means I don't have to quote as many things and usually gets a reasonable error if I did intent on matching things. But yes, for scripts you are better served by nullglob.
qiqitori 6y ago
See https://mywiki.wooledge.org/BashPitfalls https://mywiki.wooledge.org/BashPitfalls for a large collection of similar edge cases. (Recently discussed on HN: https://news.ycombinator.com/item?id=24401085 https://news.ycombinator.com/item?id=24401085)

gkfasdfasdf 6y ago

This is awesome. Question: [[ -z "${NO_COLOR-}" ]] && [[ "${TERM-}" != "dumb" ]] Is it not safe to leave off the double quotes here, because it is inside [[ ]] ?

Lt_Riza_Hawkeye 6y ago
No. I'm not sure what the "-" is doing in "${TERM-}", but if $TERM was for example set to "-z stringhere" then that would be interpreted as two arguments to "[[" instead of one, and would ensure that it always returns false, for example
- vbernat 6y ago
  The "-" is here to not error when the variable does not exist (as the topmost set is configured to trigger error on unset variables). It is meant to substitute the (empty) string on the right if the left variable does not exist.
ben509 6y ago
There's normally no need for ${var-} to get default values, and you can put && inside. I'd write that as: [[ -z $NO_COLOR && $TERM != dumb ]] As another comment pointed out, they're erroring on unset values. I don't think the line noise is worth it, but it will catch spelling mistakes, I guess.

graton 6y ago

For when things go wrong in your bash script, I really like the following: # Enable verbose logging of the bash script PS4='+ ${BASH_SOURCE:-}:${FUNCNAME[0]:-}:L${LINENO:-}: ' set -x This improves the tracing function greatly (at the cost of much higher verbosity) in that you get the filename, function, and line number in the output.

krebs_liebhaber 6y ago

> Published by Maciej Radzikowski This guy was dealt a bad hand by God.

pierrebai 6y ago

Shorter minimal safe script template: python. Seriously, give up on OS-specific, version-nightmare, quoting-hell, hairy-corner-behaviour shell scripts. I used to install Cygwin and Vim on all new Windows computers. I now install Python and VSCode on all OS. My life has never been better.

CalebJohn 6y ago

A lot of people in this thread seem to share my desire to replace some of what bash does with scripts. Which is why I wrote a utility to allow writing scripts in-between pipes [1]. The basic idea is to call this tool anytime you would be setting up a complex chain of unix pipes, and instead do the scripting in a an interactive environment that allows you to see output as you type. It's just a toy implementation right now and only supports python, but could eventually support any scripting environment (even bash). I don't really see this as a serious tool, but it can be useful for the odd shell job. Or just for those who like immediate evaluation on every key stroke ;) [1]: https://github.com/CalebJohn/shell-coupler https://github.com/CalebJohn/shell-coupler

ben509 6y ago

> This line does its best to define the script’s location directory, ~~and then we cd to it. Why?~~ (struck through in original) I've done this to make little bash utilities, and while it's cool when it works, it really doesn't work. Never try to guess the script's location. You're adding very weird, broken, magical behavior with no way to test it. So keep it simple and write these scripts to expect help from the user. If you need resources for a bash script, expect them to be in the working directory, or a standard directory, or have the user pass paths in through environment variables or other means. Yes, it's gross and inelegant. It also won't break mysteriously.

combatentropy 6y ago

Does anyone write their shell scripts in JavaScript, calling them with node?

baby 6y ago

Is there a modern alternative for bash that one can use to write bash script without having to write python? Something that looks decent if you are writing an if else, or for loop, etc.

charrondev 6y ago
Nothing that is common and built in that meets these requirements and is very legible AFAIK. I tend to write most of my scripts in PHP so I don’t have much of a leg to stand on other than that it is: - very scriptable. - common in the environments I work in. - can easily shell out. - can be either completely loosely typed or more strongly typed as you wish. - has an extensive standard library.

gurjeet 6y ago

Shameless plug; I now use Golang for all my scripting needs. I wrote about it and shared a sample here: https://news.ycombinator.com/item?id=24392172 https://news.ycombinator.com/item?id=24392172

mjgs 6y ago

I have over the years developed a similar template. It’s got usage, arg parsing, etc. One thing I find incredibly useful, which doesn’t appear in the minimal template presented here (unless I’ve missed it), is the ability to run the program in a mode where it prints out debug info like variables getting set, function names and line numbers. I add this to the start of every bash script: if [ -n "$DEBUG" ]; then echo "$0: Setting bash option -x for debug" PS4='+($(basename ${BASH_SOURCE}):${LINENO}): ${FUNCNAME[0]:+${FUNCNAME[0]}(): }' set -x fi Then to run with debug info printed just set DEBUG=1 Ideally there would be a debugger where you could step through code, but this is the closest I’ve found to that.

lenkite 6y ago

Prefer environment variables instead of command line options for bash scripts. More readable and maintainable and testable.

vietvu 6y ago

I love this template. I created one on my own but lacks parse params. However, I normally use Python script for anything needs more than 2 args.