17 ms·

The Case for Bash (2021)

125 points by memalign 3y ago

majkinetor 3y ago
Quality of the articles on the front page seems to be rapidly declining...
- Gualdrapo 3y ago
  Font alone is so thin is unreadable. It's funny to say that in HN where fonts are so small too.
  - gattr 3y ago
    On my 24" 4K monitor TFA's font was indeed verging on unreadable. HN looks fine for me, though.
msoad 3y ago
I ask chatgpt for bash scripts. So useful and every time I’m learning something new
- BoxOfRain 3y ago
  ChatGPT is great for quickly getting the bones of a script but you still need to know bash reasonably well, ChatGPT will sometimes merrily write stuff that looks correct but is actually a foot-gun and that's not great when things like `rm` are involved.
  - msoad 3y ago
    I know enough bash to be able to spot those things. but I might forget how exactly `awk` should be used to split a string or such. So asking ChatGPT is a quick way of getting those things working...
    - Danjoe4 3y ago
      ChatGPT excels at regex, awk, grep, etc.
    - sgarland 3y ago
      You won't learn unless you spend the time to try (and fail) doing so yourself.
      - msoad 3y ago
        You can't tell me how I function. I'm def learning new neat tricks asking things from ChatGTP. I'm 100% sure if you paste your bash scripts in there and ask for suggestion you'll learn too
pjc50 3y ago
The only real advantage of bash (or other shells) is the ability to build up to it from the command line; to use it as a REPL then embed the stuff you've just done into a script. The massive disadvantage of bash is that it uses a legal filename character " " as a delimiter ("$IFS"). That means it's very easy to write scripts which work fine on your test case but blow up in a different context. And the worst case of that is accidentally deleting the user's file system. So many of the cute little bash examples you'll see, including I think the awk one on this page, will go wrong if you have a space in the wrong place in your filesystem. Or, god help you, a newline. Or a file named "*". Or "--".
- Joker_vD 3y ago
  Quotes, dot-slash, quotes, double dash and quotes. Quotes, quotes, shellcheck and quotes. Quotes, quotes, quotes, lovely quotes! Just don't write bugs, what's so hard about shell programming?
  - EuAndreh 3y ago
    Just shellcheck is enough.
    - ndsipa_pomu 3y ago
      Shellcheck won't catch the missing dashes ('--') at the end of command options, so you could be in trouble if a variable starts with a dash and the command interprets it as an option rather than the filename. It's not particularly obvious, but if people can upload a file and specify its name, then they could compromise a script by choosing a suitably evil name. If you get into a habit of putting '--' at the end of the options and before the filename variable, then you protect against that.
- JdeBP 3y ago
  Or, put another way: Getting the 10% of the cases that have whitespace or metacharacters in variable values and filenames to work ... takes the other 90% of the time. (-: I habitually quote variable expansions, even when I know that the variables are never going to contain whitespace. Because long experience has taught me that one day they will. Just like I've learned never to assume that ${PATH} is always non-blank and so assume that PATH="${PATH}:/something" won't accidentally add the current directory to my search path. (-:
jagged-chisel 3y ago
> It coordinates programs… Sounds like a case for shells in general. No other script-execution tools make running other programs (and linking their inputs and outputs) so simple. To overcome Bash’s problems, one can choose another shell. But I suppose you can’t just pick your personal favorite and script in its dialect, because then how would the human tasked with integrating the script into $wherever know which shells to have installed? You can stick with sh which is guaranteed to be installed, or bash (indeed an improvement over sh) which is nearly guaranteed to be installed.
- mrspuratic 3y ago
  More than a decade ago I needed a multi-platform way to manage Nagios client data collection (mostly) using whatever native tools were available, so I wrote one. bash, gawk and send_nsca were the only 3 binaries needed, and it ran well on 10 different *nixs (free and commercial) on Alpha, ARM, MIPS, SPARC, X86 and X86_64 platforms. Of course, after a decade of incremental tinkering I ended up with exactly what you anecdon't want: ~2500 lines of shell script and ~1000 lines of awk... LOC misleads of course, there were 15 modular plugins, so there was a lot of boilerplate, and this includes non-runtime ~1000 lines for install, self-test and sanity checks.
- JdeBP 3y ago
  Or one can learn the lessons that Debian learned over a decade ago, and Ubuntu almost two decades ago, about bashisms, maintainability, and performance; and script in at most the Debian Almquist shell, whatever fancy shell one may be using interactively. * https://wiki.ubuntu.com/DashAsBinSh https://wiki.ubuntu.com/DashAsBinSh * https://wiki.ubuntu.com/DashAsBinSh/Spec https://wiki.ubuntu.com/DashAsBinSh/Spec * https://wiki.debian.org/BootProcessSpeedup#Using_a_faster_system_shell https://wiki.debian.org/BootProcessSpeedup#Using_a_faster_sy... * https://lists.debian.org/debian-release/2007/07/msg00027.html https://lists.debian.org/debian-release/2007/07/msg00027.htm... * https://lwn.net/Articles/343924/ https://lwn.net/Articles/343924/ * https://www.debian.org/doc/manuals/debian-reference/ch12.en.html#_posix_shell_compatibility https://www.debian.org/doc/manuals/debian-reference/ch12.en....
  - 3y ago
JdeBP 3y ago
The article is a bit confused about whether it's making a case for shell scripting in general, or for the Bourne Again shell in particular. By the looks of things, its target readership is people who don't use shell scripting, or interactive shells, at all; so the former case would be more appropriate, and the drawing of a distinction between the Bourne Again, Friendly Interactive, Z, and other shells just muddies the waters for that readership.
cwingrav 3y ago
BASH is like Obi Wan. It isn’t the most powerful or flashiest, but it survived a long time, where others didn’t, for very good reasons. Bash runs basically everywhere. It has many modern features you wouldn’t expect. Its syntax is literally what you would type on the command line if you were diagnosing or fixing systems so you don’t need to transpile to another language. Its reliance on other programs means it is glue and can easily incorporate highly cohesive functionality/tools others write and maintain. Also, it’s been around and is everywhere so you don’t worry about trying to incorporate the current latest and greatest declarative tool (which will blow over in 5 years) into your other workflows. Basically, don’t disparage a Jedi/tool that has survived where others didn’t. There is a reason.
- cwingrav 3y ago
  Also, use shellcheck. Incorporate it into you editor. Fix all warning and don’t ignore them. This will push you deep into bash syntax rabbit holes but you come out better the other side.
  - ndsipa_pomu 3y ago
    Also, familiarise yourself with https://mywiki.wooledge.org/BashPitfalls https://mywiki.wooledge.org/BashPitfalls
  - blueflow 3y ago
    Find the problem: echo "$(tr -dc A-Za-z0-9 </dev/urandom | dd count=1 bs=16 2>/dev/null)" ^-- SC2005 (style): Useless echo? Instead of 'echo $(cmd)', just use 'cmd'. I have experienced so many situations where shellcheck is giving harmful advice or warns me about the thing that is exactly my intention.
    - saagarjha 3y ago
      …what’s the problem?
    - ndsipa_pomu 3y ago
      Using "echo" at all is a problem. It's recommended to switch to "printf" instead as echo has unfixable problems, especially with strings that start with a dash. printf "%s\n" "$(tr -dc A-Za-z0-9 </dev/urandom | dd count=1 bs=16 2>/dev/null)" If you don't require the line feed, then you can just use: tr -dc A-Za-z0-9 </dev/urandom | dd count=1 bs=16 2>/dev/null
deafpolygon 3y ago
Bash is a shell. Learn the shell, and you "learn" bash.
- DonHopkins 3y ago
  At the risk of being accused of bash bashing, I'll throw down that bash is a weak ineffective shell. A "shell" can and should be so much more than all bash has to offer. The much earlier more powerful ITS (Incompatible Timesharing System) shell DDT (whose job name was HACTRN) had an integrated PDP-10 machine language debugger / assembler / disassembler, so you could interactively or batch "script" and patch DDT and even other jobs in full blown PDP-10 assembly language. Why invent yet another half assed "scripting" language that no other jobs in the system are using, when you can use the same fully powerful machine language that every job in the system is using? And if you need to do anything complicated, there's always LISP! DDT's syntax was even more obscure than bash (and only a bit less obscure than TECO), but it was much more powerful and elegant, unleashing the full undiluted power of the PDP-10 at your fingertips. https://github.com/PDP-10/its/blob/master/doc/_info_/ddtord.1462 https://github.com/PDP-10/its/blob/master/doc/_info_/ddtord.... You could also write DDT commands in text files like your login file, i.e. assembling a few lines of code to print the prompt by making a system call to get the time and format it as text. You could examine and deposit code and data, load and change symbols, set breakpoints, in your own and even other user's running jobs (processes)! You could disconnect without logging out (accidentally or not) and all your jobs would stay around until you logged back in, at which time you could reattach your job tree and continue what you were doing, all without running something like "screen" -- that crucial feature was just built in and always worked. You could also pass ownership of jobs (like a running ZORK game or LISP interpreter or FOOBAR feeper) back and forth between users to share, like passing a joint. ;) https://news.ycombinator.com/item?id=22840639 https://news.ycombinator.com/item?id=22840639 >It helped that ITS had no security whatsoever! But it had some very obscure commands, like $$^R (literally: two escapes followed by a control-R). >There was an obscure symbol that went with it called "DPSTOK" ("DePoSiT OK", presumably) that, if you set it to -1, allowed you to type $$^R to mess with other people's jobs, dynamically patch their code, etc. (The DDT top level shell had a built-in assembler/debugger, and anyone could read anybody else's job's memory, but you needed to use $$^R to enable writing). http://www.poppyfields.net/filks/00117.html http://www.poppyfields.net/filks/00117.html The HACTRN Original song: The Raven Original artist: Edgar Allan Poe Filk author: Guy L. Steele Jr. Intro: Notes for those not familar with the terms in this poem -- see below [...] DDT ("dee dee tee") HACTRN ("hack-tran") = top level debugging and job controlling procedure, capable of controlling up to eight simultaneous jobs (which may themselves be DDTs!) and performing other miscellaneous functions. HACTRN specifically denotes a DDT at the top of a job tree, while DDT is the more general term. The two terms refer to the same job in the poem, and are thus treated as synonymous. Note that DDT requires its subjobs to have unique names for obvious reasons; hence the concern over seven jobs all named FOO.
augustk 3y ago
This article is not specifically about Bash, but about the shell command language sh. The POSIX Shell is a standardized subset of Bash which has even better portability. I would recommend learning and using sh unless you really need Bash. https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V... Your script will use sh if it starts with #!/bin/sh (instead of #!/bin/bash). I can also recommend ShellCheck which is a shell script analysis tool (implemented in Haskell) which can find errors and potential problems in your scripts. On a Debian system, installing ShellScheck is as simple as `sudo apt install shellcheck'.
- spyremeown 3y ago
  >I would recommend learning and using sh unless you really need Bash. I don't buy this argument. Ok, it's a standard, but sometimes it's a pain-in-the-hole standard. Bash augmentations lessen some of the pains, and it's just nicer. The only situation is if you're using busybox in a very limited system or you reeeeeally need your script to run on many difference unices, which let's be honest, is not that common nowadays.
  - JdeBP 3y ago
    All of this was covered in great detail in the 2000s, when Debian and Ubuntu switched /bin/sh to the Debian Almquist shell, which was basically POSIX-only with some 3 things that Debian people simply couldn't live without, and encouraged system scripts to use it. If you weren't around then, go and read the discussions. They're mostly still available, and they cover some important stuff that straw man counterarguments regularly miss. The Debian people were concerned, for starters, with how much time the Bourne Again shell spent, at process initialization, setting up things for extensions and interactive features that were never employed in non-interactive "sh" mode; a significant cause for concern given how much of the system was executable shell scripts.
    - jmclnx 3y ago
      >with some 3 things that Debian people simply couldn't live without Curious, what are the 3 things ? But yes, people should always use sh (or ksh) for scripting as opposed to bash, why, it is far more portable to other systems.
DonHopkins 3y ago
The only thing you need Bash for is to install Python.
BirAdam 3y ago
I’ve written a ridiculous number of shell scripts to automate many many things. It’s great for doing something now, and it’s great as a way to interact with a machine. It is not great in all cases. Particularly: array handling is bad, it’s slow if you start using actual Bashisms, very few younger devs seem to know it. The shell is great cuz it’s there and it can cover about 50% of use cases for server side automation, but it’s bad for the reasons mentioned above. I love it and I hate it.
frumiousirc 3y ago
Am I missing something or is the article's central example using `tail` fundamentally broken? The `sort` blocks until input is closed yet `tail -f` never closes its output.
- deleted 3y ago
  [deleted]
- JdeBP 3y ago
  You'll be pointing out that [ is a built-in command, next. (-:
  - rascul 3y ago
    It's also a binary in /bin (or a symlink because of /usr merge) and it's silly that the article tries to view [ in a pager.
- sgarland 3y ago
  Not to mention, they're under-using awk. Tailing this seems completely unnecessary. The original from TFA: `tail -fq /var/log/nginx/access.log | awk '{print $7}' | sort | uniq -c | sort -rn` can be replaced with: `awk '{ a[$7]++ } END { for (i in a) print " " a[i] " " i }' /var/log/nginx/access.log | sort -rn` or, if you have GNU awk: `awk '{ a[$7]++ } END { PROCINFO["sorted_in"] = "@val_num_desc"; for (i in a) print " " a[i] " " i }' /var/log/nginx/access.log`
_8j50 3y ago
I feel like zsh is for pros maybe. I use Linux daily and write python and shellscripts. Zsh has been defaulted all over the place but all that meant for me is I have to figure out how to get it to do things the bash way once in a while. I see no reason why distros like ubuntu or kali would default to zsh.I use the shell for sysadmin stuff, anything complex or app specific gets at least a python treatment. The whole thing reminds me of python 2->3 or sysv init to systemd. There must be people somewhere deeply invested in these changes and their voice is certainly much more valuable than the average opensource joe user. Defaults matter and the time I and millions had to spend learning to transition for no direct benefit to us has value. The whole distro model of everyone getting a distro they like falls apart when not enough people can fork. Overall, this devalues opensource as an investment. Because of unpredictable hidden costs like this.
pgtan 3y ago
Well, Kornshell (ksh93) has some nice programming features, much better than bash.
lloeki 3y ago
> The Python and Ruby versions would be similar Not Ruby! sh = Shell.cd('/') sh.transact do system('tail -fq /var/log/nginx/access.log') | system("awk '{print $7}" | system("sort") | system("uniq -c") | system("sort -rn") end Pretty sure one could get without the system method with Shell having def method_missing(meth, * args) to call system([meth.to_s, * args]) or something, turning it into: sh.transact do tail '-fq' '/var/log/nginx/access.log' | awk '{print $7}' | sort | uniq '-c' | sort '-rn' end https://github.com/ruby/shell#pipe-etcprintcap-into-a-file https://github.com/ruby/shell#pipe-etcprintcap-into-a-file
rfittich 3y ago
I think this perspective is essential in our ever-evolving tech industry. Bash, despite its quirks and often steep learning curve, has its niche in our toolset. Its power lies in its simplicity and direct access to system level functions, which makes it an efficient tool for system administrators and developers working in DevOps. It's excellent for automating small to medium-sized tasks on UNIX-like systems where the overhead of a full-fledged language would be overkill. While it's true that bash has its limitations, the value of its ubiquity cannot be overstated. The bash shell is everywhere - from massive server clusters to tiny embedded systems. This means that a bash script written on one system is very likely to work unchanged on another.
- csdvrx 3y ago
  > the value of its ubiquity cannot be overstated TBH it's among the many reasons I've return to bash from zsh I want to experience first hand the limitations I may hit on other servers. I also try to use #!/bin/ash in my scripts to live life in hard mode :)
666satanhimself 3y ago
[dead]
todd8 3y ago
I'm a computer scientist/software engineer that has used primarily Unix or Linux systems since the 80's. I write shell scripts occasionally, but I just find them error prone enough that I'd rather use python than sh for complex scripts. Python has some packages that make working with files, directories, and processes tolerable and at least I can understand the string quoting rules in python. Generally my order of preference for shell scripting tasks is: python > bash > pearl > tcl > sh > anything > AppleTalk.
- nathanmcrae 3y ago
  What packages would you recommend for this kind of usage?
  - bustermellotron 3y ago
    This covers most of what you need for "housekeeping" scripts in Python: https://automatetheboringstuff.com https://automatetheboringstuff.com
BoardsOfCanada 3y ago
Maybe this already exists, but I imagine if someone wrote at library for interacting with gnu or posix commands from python (or ruby, or...) that built up the commands in a structured way and parsed the replies and put them in a dict, that would be very nice to use instead of bash-scripts. Every weird thing that you wanted to do would be a lot easier to do in python than in bash, and none of that looking out for spaces in strings and so on.
- meta-meta 3y ago
  https://github.com/babashka/babashka https://github.com/babashka/babashka comes to mind
- throwaway858 3y ago
  There is this for Haskell, which automatically converts all shell commands (like "ls") to regular Haskell functions: https://chrisdone.com/posts/shell-conduit/ https://chrisdone.com/posts/shell-conduit/ It uses a Haskell streaming library so you can do stuff like shell pipelines and file redirection (but in a more structured, safer and more powerful way)
- Too 3y ago
  jc is the most comprehensive I’m aware of. https://kellyjonbrazil.github.io/jc/ https://kellyjonbrazil.github.io/jc/ Except a lot of the times you don’t need this. The standard library may have what you need. There is no reason to pipe ls through jc, when you could use os.listdir().
faeriechangling 3y ago
The thing that makes me loathe Bash more than anything else is it’s 3 types are string, int, and list and those are not the only types of data I work with day to day.
vram22 3y ago
IMHO, the best resource to learn about both Unix shell usage at the command-line prompt, and shell programming in .sh script files, is still the book "The Unix Programming Environment" by Kernighan and Pike, except for shell gotchas and subtleties which were discovered later. Of course, they still would not have covered all possible issues known even at that time, because that was not the focus of the book, which was to be a Unix tutorial, not just on shell, but many other commands, general Unix usage, and the environment too (hence the title of the book). But they did cover and warn about some issues, including giving solutions.
SomeCallMeTim 3y ago
Bash/Sh is an objectively awful programming language. You could likely fill an entire book with all of the design fails that is Bash script syntax. Only in a shell script would you ever need eight backslash characters in a row in order to accomplish something...and "space as delimiter" is awful in so many ways. But especially Sh runs everywhere, so it's good to know. This seems to be the article's point. Honestly, though? The right answer is to transpile a better language to Sh. Looking around, there have been several half-hearted, abandoned attempts. But apparently everyone is happy to just fall back on the "accidental syntax" of Bash that really doesn't make sense when compared to ... any other language. I put up with Bash because there's no better alternative. That doesn't mean we should put up with Bash; just that we must put up with it at least to the point where it can run code written in a real language.
- kagevf 3y ago
  > "space as delimiter" is awful in so many ways Lisp and Scheme would like to have a word . . .
  - deleted 3y ago
    [deleted]
  - SomeCallMeTim 3y ago
    I needed to learn Lisp in college. I hate Lisp. 'nough said.
- danielvaughn 3y ago
  I'm not very knowledgeable in this area as I don't do much shell scripting, but isn't zsh supposed to be some kind of enhancement over bash? Does it suffer from the same issues? Regardless, the idea of a language that transpiles to bash is kinda interesting.
  - NERD_ALERT 3y ago
    ZSH is basically the same thing. For all intents and purposes it is basically the same thing. The best attempt I’ve seen to truly make a better shell is Nushell. https://www.nushell.sh/ https://www.nushell.sh/
jquast 3y ago
We should have better local shells that “drive” remote sh/bash shells through transpilation, so that we can have modern shells anywhere without concern for remote install, access to install or compile, compatibility, carrying customizations, etc.
SPBS 3y ago
So it's both ubiquitous and a terse DSL for managing and coordinating programs. Yeah I agree. Also if you're on Windows it's nice that PowerShell is always preinstalled and serves the same purpose.
stanleydrew 3y ago
This argument isn't very compelling. Shell scripts are great at tricking you into believing you've got something 100% working when in fact you've only accounted for the happy path and maybe two or three failure cases. This is the primary reason shell scripts are so often broken. There are tons of assumptions baked in which are easily violated. If your shell script keeps breaking, and you keep fixing it by writing more shell, then you're in an abusive relationship with your programming environment.
- nickm12 3y ago
  "Shell scripts are great at tricking you into believing you've got something 100% working" Exactly. Shell scripting is basically a non-hygenic macro language, with very complex semantics around its primary data type, the string. It also lacks pretty much all of the affordances we use to write reliable programs. I have no doubt someone has written a typechecker or unit test framework for bash, but I've never seen one used. I'll take the ugly code with the straightforward semantics, thanks.
shmerl 3y ago
Good points. But I wish Bash had better syntax and built in support for basic concepts like boolean logic. Try to compose some complex boolean expression in Bash and you will quickly hit problems.
sigmonsays 3y ago
it seems like most people that have dislike bash never spent the time to learn it. I have a love/hate relationship with is. I started with computers in 2000 and wrote a ton of bash for various things. Compiling software, running cicd, adhoc batch jobs, etc. I've managed many cicd pipelines with bash in my time. While it's not great, once you know the basics, you can be very successful. That being said, rewriting the bash "program" in another language is much advised. If your bash is over a few hundred lines, chances are you've outgrown it Some things in bash (mainly pipes and redirection) are just too easy. Job control is also great. The 'process' and 'shell script' are two primitives that make a lot possible. Trying to do pipes or redirection in python is awful. There are a ton of subtle bugs you have to worry about
- bluepizza 3y ago
  Seems like you spent the time to learn it and still dislike it?
  - sigmonsays 3y ago
    hope it didn't seem like I dislike it. It's just a tool for a job. I still write bash and love it for the simple things. Now I have the experience to rewrite it early before it gets out of hand.
makz 3y ago
I have a counterpoint for many comments here. If you are writing some Python for example, and most of your code is executing commands such as tail, cat, wc, etc. and then doing something with their output, you are better off writing a shell script instead .
- DerekL 3y ago
  But if you're in Python, why would you need to need to execute separate processes to do any of that?
  - makz 3y ago
    I don’t know, ask the people writing that kind of stuff.
    - Leo_Germond 3y ago
      In my own experience people typically do that when they have some experience in bash, none in python, and try to do something in python the way they would do it in bash. So for those my advice, instead of getting back again to bash is to get a bit curious about the python standard lib, which has a nice doc unlinke bash, and not assume they know everything about scripting just because they have spent a long amount of time writing bash script.
      - rascul 3y ago
        > which has a nice doc unlinke bash I find the bash reference manual to be nice enough for me. https://www.gnu.org/savannah-checkouts/gnu/bash/manual/html_node/index.html https://www.gnu.org/savannah-checkouts/gnu/bash/manual/html_...
jononomo 3y ago
I have found that GPT-4 can write bash scripts for me pretty well.
andrewl 3y ago
The article recommends Bite Size Bash by Julia Evans. Does anybody have an experience with it?
- asicsp 3y ago
  You could check out a sample of her comics here: https://wizardzines.com/comics/ https://wizardzines.com/comics/ (there's a search box too). Her articles often hit HN front page: https://news.ycombinator.com/from?site=jvns.ca https://news.ycombinator.com/from?site=jvns.ca
owenpalmer 3y ago
1. Someone creates a platform. 2. They make a terrible language the most convenient language. 3. People use the platform, along with the terrible language. 4. This terrible language is now everywhere. 5. The argument for using this terrible language is #4
gtsop 3y ago
As a serious programmer, I have printed out and read most of the bash v5 manual. As a serious programmer, i do TDD even with bash, it is called bats-core As a serious programer I use the right tool for the right job, so I write bash scripts when it makes sense. Having done all these, I rearly write bash programs, but when I do, it is a joy! Plus i have gteater terminal understanding which is useful beyond bash programming.
amadeuspagel 3y ago
> This is a contrived example Indeed, in JS you wouldn't use awk and sort, you'd use JS array functions. It's fine to have examples that are contrived in the sense that the problem is contrived, but the solution should not be.
leke 3y ago
I was just thinking about whether or not there is a compile to bash language.

125 points by memalign 3y ago

majkinetor 3y ago

Quality of the articles on the front page seems to be rapidly declining...

Gualdrapo 3y ago
Font alone is so thin is unreadable. It's funny to say that in HN where fonts are so small too.
- gattr 3y ago
  On my 24" 4K monitor TFA's font was indeed verging on unreadable. HN looks fine for me, though.

msoad 3y ago

I ask chatgpt for bash scripts. So useful and every time I’m learning something new

BoxOfRain 3y ago
ChatGPT is great for quickly getting the bones of a script but you still need to know bash reasonably well, ChatGPT will sometimes merrily write stuff that looks correct but is actually a foot-gun and that's not great when things like `rm` are involved.
- msoad 3y ago
  I know enough bash to be able to spot those things. but I might forget how exactly `awk` should be used to split a string or such. So asking ChatGPT is a quick way of getting those things working...
  - Danjoe4 3y ago
    ChatGPT excels at regex, awk, grep, etc.
  - sgarland 3y ago
    You won't learn unless you spend the time to try (and fail) doing so yourself.
    - msoad 3y ago
      You can't tell me how I function. I'm def learning new neat tricks asking things from ChatGTP. I'm 100% sure if you paste your bash scripts in there and ask for suggestion you'll learn too

pjc50 3y ago

The only real advantage of bash (or other shells) is the ability to build up to it from the command line; to use it as a REPL then embed the stuff you've just done into a script. The massive disadvantage of bash is that it uses a legal filename character " " as a delimiter ("$IFS"). That means it's very easy to write scripts which work fine on your test case but blow up in a different context. And the worst case of that is accidentally deleting the user's file system. So many of the cute little bash examples you'll see, including I think the awk one on this page, will go wrong if you have a space in the wrong place in your filesystem. Or, god help you, a newline. Or a file named "*". Or "--".

Joker_vD 3y ago
Quotes, dot-slash, quotes, double dash and quotes. Quotes, quotes, shellcheck and quotes. Quotes, quotes, quotes, lovely quotes! Just don't write bugs, what's so hard about shell programming?
- EuAndreh 3y ago
  Just shellcheck is enough.
  - ndsipa_pomu 3y ago
    Shellcheck won't catch the missing dashes ('--') at the end of command options, so you could be in trouble if a variable starts with a dash and the command interprets it as an option rather than the filename. It's not particularly obvious, but if people can upload a file and specify its name, then they could compromise a script by choosing a suitably evil name. If you get into a habit of putting '--' at the end of the options and before the filename variable, then you protect against that.
JdeBP 3y ago
Or, put another way: Getting the 10% of the cases that have whitespace or metacharacters in variable values and filenames to work ... takes the other 90% of the time. (-: I habitually quote variable expansions, even when I know that the variables are never going to contain whitespace. Because long experience has taught me that one day they will. Just like I've learned never to assume that ${PATH} is always non-blank and so assume that PATH="${PATH}:/something" won't accidentally add the current directory to my search path. (-:

jagged-chisel 3y ago

> It coordinates programs… Sounds like a case for shells in general. No other script-execution tools make running other programs (and linking their inputs and outputs) so simple. To overcome Bash’s problems, one can choose another shell. But I suppose you can’t just pick your personal favorite and script in its dialect, because then how would the human tasked with integrating the script into $wherever know which shells to have installed? You can stick with sh which is guaranteed to be installed, or bash (indeed an improvement over sh) which is nearly guaranteed to be installed.

mrspuratic 3y ago
More than a decade ago I needed a multi-platform way to manage Nagios client data collection (mostly) using whatever native tools were available, so I wrote one. bash, gawk and send_nsca were the only 3 binaries needed, and it ran well on 10 different *nixs (free and commercial) on Alpha, ARM, MIPS, SPARC, X86 and X86_64 platforms. Of course, after a decade of incremental tinkering I ended up with exactly what you anecdon't want: ~2500 lines of shell script and ~1000 lines of awk... LOC misleads of course, there were 15 modular plugins, so there was a lot of boilerplate, and this includes non-runtime ~1000 lines for install, self-test and sanity checks.
JdeBP 3y ago
Or one can learn the lessons that Debian learned over a decade ago, and Ubuntu almost two decades ago, about bashisms, maintainability, and performance; and script in at most the Debian Almquist shell, whatever fancy shell one may be using interactively. * https://wiki.ubuntu.com/DashAsBinSh https://wiki.ubuntu.com/DashAsBinSh * https://wiki.ubuntu.com/DashAsBinSh/Spec https://wiki.ubuntu.com/DashAsBinSh/Spec * https://wiki.debian.org/BootProcessSpeedup#Using_a_faster_system_shell https://wiki.debian.org/BootProcessSpeedup#Using_a_faster_sy... * https://lists.debian.org/debian-release/2007/07/msg00027.html https://lists.debian.org/debian-release/2007/07/msg00027.htm... * https://lwn.net/Articles/343924/ https://lwn.net/Articles/343924/ * https://www.debian.org/doc/manuals/debian-reference/ch12.en.html#_posix_shell_compatibility https://www.debian.org/doc/manuals/debian-reference/ch12.en....
- 3y ago

JdeBP 3y ago

The article is a bit confused about whether it's making a case for shell scripting in general, or for the Bourne Again shell in particular. By the looks of things, its target readership is people who don't use shell scripting, or interactive shells, at all; so the former case would be more appropriate, and the drawing of a distinction between the Bourne Again, Friendly Interactive, Z, and other shells just muddies the waters for that readership.

cwingrav 3y ago

BASH is like Obi Wan. It isn’t the most powerful or flashiest, but it survived a long time, where others didn’t, for very good reasons. Bash runs basically everywhere. It has many modern features you wouldn’t expect. Its syntax is literally what you would type on the command line if you were diagnosing or fixing systems so you don’t need to transpile to another language. Its reliance on other programs means it is glue and can easily incorporate highly cohesive functionality/tools others write and maintain. Also, it’s been around and is everywhere so you don’t worry about trying to incorporate the current latest and greatest declarative tool (which will blow over in 5 years) into your other workflows. Basically, don’t disparage a Jedi/tool that has survived where others didn’t. There is a reason.

cwingrav 3y ago
Also, use shellcheck. Incorporate it into you editor. Fix all warning and don’t ignore them. This will push you deep into bash syntax rabbit holes but you come out better the other side.
- ndsipa_pomu 3y ago
  Also, familiarise yourself with https://mywiki.wooledge.org/BashPitfalls https://mywiki.wooledge.org/BashPitfalls
- blueflow 3y ago
  Find the problem: echo "$(tr -dc A-Za-z0-9 </dev/urandom | dd count=1 bs=16 2>/dev/null)" ^-- SC2005 (style): Useless echo? Instead of 'echo $(cmd)', just use 'cmd'. I have experienced so many situations where shellcheck is giving harmful advice or warns me about the thing that is exactly my intention.
  - saagarjha 3y ago
    …what’s the problem?
  - ndsipa_pomu 3y ago
    Using "echo" at all is a problem. It's recommended to switch to "printf" instead as echo has unfixable problems, especially with strings that start with a dash. printf "%s\n" "$(tr -dc A-Za-z0-9 </dev/urandom | dd count=1 bs=16 2>/dev/null)" If you don't require the line feed, then you can just use: tr -dc A-Za-z0-9 </dev/urandom | dd count=1 bs=16 2>/dev/null

deafpolygon 3y ago

Bash is a shell. Learn the shell, and you "learn" bash.

DonHopkins 3y ago
At the risk of being accused of bash bashing, I'll throw down that bash is a weak ineffective shell. A "shell" can and should be so much more than all bash has to offer. The much earlier more powerful ITS (Incompatible Timesharing System) shell DDT (whose job name was HACTRN) had an integrated PDP-10 machine language debugger / assembler / disassembler, so you could interactively or batch "script" and patch DDT and even other jobs in full blown PDP-10 assembly language. Why invent yet another half assed "scripting" language that no other jobs in the system are using, when you can use the same fully powerful machine language that every job in the system is using? And if you need to do anything complicated, there's always LISP! DDT's syntax was even more obscure than bash (and only a bit less obscure than TECO), but it was much more powerful and elegant, unleashing the full undiluted power of the PDP-10 at your fingertips. https://github.com/PDP-10/its/blob/master/doc/_info_/ddtord.1462 https://github.com/PDP-10/its/blob/master/doc/_info_/ddtord.... You could also write DDT commands in text files like your login file, i.e. assembling a few lines of code to print the prompt by making a system call to get the time and format it as text. You could examine and deposit code and data, load and change symbols, set breakpoints, in your own and even other user's running jobs (processes)! You could disconnect without logging out (accidentally or not) and all your jobs would stay around until you logged back in, at which time you could reattach your job tree and continue what you were doing, all without running something like "screen" -- that crucial feature was just built in and always worked. You could also pass ownership of jobs (like a running ZORK game or LISP interpreter or FOOBAR feeper) back and forth between users to share, like passing a joint. ;) https://news.ycombinator.com/item?id=22840639 https://news.ycombinator.com/item?id=22840639 >It helped that ITS had no security whatsoever! But it had some very obscure commands, like $$^R (literally: two escapes followed by a control-R). >There was an obscure symbol that went with it called "DPSTOK" ("DePoSiT OK", presumably) that, if you set it to -1, allowed you to type $$^R to mess with other people's jobs, dynamically patch their code, etc. (The DDT top level shell had a built-in assembler/debugger, and anyone could read anybody else's job's memory, but you needed to use $$^R to enable writing). http://www.poppyfields.net/filks/00117.html http://www.poppyfields.net/filks/00117.html The HACTRN Original song: The Raven Original artist: Edgar Allan Poe Filk author: Guy L. Steele Jr. Intro: Notes for those not familar with the terms in this poem -- see below [...] DDT ("dee dee tee") HACTRN ("hack-tran") = top level debugging and job controlling procedure, capable of controlling up to eight simultaneous jobs (which may themselves be DDTs!) and performing other miscellaneous functions. HACTRN specifically denotes a DDT at the top of a job tree, while DDT is the more general term. The two terms refer to the same job in the poem, and are thus treated as synonymous. Note that DDT requires its subjobs to have unique names for obvious reasons; hence the concern over seven jobs all named FOO.

augustk 3y ago

This article is not specifically about Bash, but about the shell command language sh. The POSIX Shell is a standardized subset of Bash which has even better portability. I would recommend learning and using sh unless you really need Bash. https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V... Your script will use sh if it starts with #!/bin/sh (instead of #!/bin/bash). I can also recommend ShellCheck which is a shell script analysis tool (implemented in Haskell) which can find errors and potential problems in your scripts. On a Debian system, installing ShellScheck is as simple as `sudo apt install shellcheck'.

spyremeown 3y ago
>I would recommend learning and using sh unless you really need Bash. I don't buy this argument. Ok, it's a standard, but sometimes it's a pain-in-the-hole standard. Bash augmentations lessen some of the pains, and it's just nicer. The only situation is if you're using busybox in a very limited system or you reeeeeally need your script to run on many difference unices, which let's be honest, is not that common nowadays.
- JdeBP 3y ago
  All of this was covered in great detail in the 2000s, when Debian and Ubuntu switched /bin/sh to the Debian Almquist shell, which was basically POSIX-only with some 3 things that Debian people simply couldn't live without, and encouraged system scripts to use it. If you weren't around then, go and read the discussions. They're mostly still available, and they cover some important stuff that straw man counterarguments regularly miss. The Debian people were concerned, for starters, with how much time the Bourne Again shell spent, at process initialization, setting up things for extensions and interactive features that were never employed in non-interactive "sh" mode; a significant cause for concern given how much of the system was executable shell scripts.
  - jmclnx 3y ago
    >with some 3 things that Debian people simply couldn't live without Curious, what are the 3 things ? But yes, people should always use sh (or ksh) for scripting as opposed to bash, why, it is far more portable to other systems.

DonHopkins 3y ago

The only thing you need Bash for is to install Python.

BirAdam 3y ago

I’ve written a ridiculous number of shell scripts to automate many many things. It’s great for doing something now, and it’s great as a way to interact with a machine. It is not great in all cases. Particularly: array handling is bad, it’s slow if you start using actual Bashisms, very few younger devs seem to know it. The shell is great cuz it’s there and it can cover about 50% of use cases for server side automation, but it’s bad for the reasons mentioned above. I love it and I hate it.

frumiousirc 3y ago

Am I missing something or is the article's central example using `tail` fundamentally broken? The `sort` blocks until input is closed yet `tail -f` never closes its output.

deleted 3y ago
[deleted]
JdeBP 3y ago
You'll be pointing out that [ is a built-in command, next. (-:
- rascul 3y ago
  It's also a binary in /bin (or a symlink because of /usr merge) and it's silly that the article tries to view [ in a pager.
sgarland 3y ago
Not to mention, they're under-using awk. Tailing this seems completely unnecessary. The original from TFA: `tail -fq /var/log/nginx/access.log | awk '{print $7}' | sort | uniq -c | sort -rn` can be replaced with: `awk '{ a[$7]++ } END { for (i in a) print " " a[i] " " i }' /var/log/nginx/access.log | sort -rn` or, if you have GNU awk: `awk '{ a[$7]++ } END { PROCINFO["sorted_in"] = "@val_num_desc"; for (i in a) print " " a[i] " " i }' /var/log/nginx/access.log`

_8j50 3y ago

I feel like zsh is for pros maybe. I use Linux daily and write python and shellscripts. Zsh has been defaulted all over the place but all that meant for me is I have to figure out how to get it to do things the bash way once in a while. I see no reason why distros like ubuntu or kali would default to zsh.I use the shell for sysadmin stuff, anything complex or app specific gets at least a python treatment. The whole thing reminds me of python 2->3 or sysv init to systemd. There must be people somewhere deeply invested in these changes and their voice is certainly much more valuable than the average opensource joe user. Defaults matter and the time I and millions had to spend learning to transition for no direct benefit to us has value. The whole distro model of everyone getting a distro they like falls apart when not enough people can fork. Overall, this devalues opensource as an investment. Because of unpredictable hidden costs like this.

pgtan 3y ago

Well, Kornshell (ksh93) has some nice programming features, much better than bash.

lloeki 3y ago

> The Python and Ruby versions would be similar Not Ruby! sh = Shell.cd('/') sh.transact do system('tail -fq /var/log/nginx/access.log') | system("awk '{print $7}" | system("sort") | system("uniq -c") | system("sort -rn") end Pretty sure one could get without the system method with Shell having def method_missing(meth, * args) to call system([meth.to_s, * args]) or something, turning it into: sh.transact do tail '-fq' '/var/log/nginx/access.log' | awk '{print $7}' | sort | uniq '-c' | sort '-rn' end https://github.com/ruby/shell#pipe-etcprintcap-into-a-file https://github.com/ruby/shell#pipe-etcprintcap-into-a-file

rfittich 3y ago

I think this perspective is essential in our ever-evolving tech industry. Bash, despite its quirks and often steep learning curve, has its niche in our toolset. Its power lies in its simplicity and direct access to system level functions, which makes it an efficient tool for system administrators and developers working in DevOps. It's excellent for automating small to medium-sized tasks on UNIX-like systems where the overhead of a full-fledged language would be overkill. While it's true that bash has its limitations, the value of its ubiquity cannot be overstated. The bash shell is everywhere - from massive server clusters to tiny embedded systems. This means that a bash script written on one system is very likely to work unchanged on another.

csdvrx 3y ago
> the value of its ubiquity cannot be overstated TBH it's among the many reasons I've return to bash from zsh I want to experience first hand the limitations I may hit on other servers. I also try to use #!/bin/ash in my scripts to live life in hard mode :)

666satanhimself 3y ago

[dead]

todd8 3y ago

I'm a computer scientist/software engineer that has used primarily Unix or Linux systems since the 80's. I write shell scripts occasionally, but I just find them error prone enough that I'd rather use python than sh for complex scripts. Python has some packages that make working with files, directories, and processes tolerable and at least I can understand the string quoting rules in python. Generally my order of preference for shell scripting tasks is: python > bash > pearl > tcl > sh > anything > AppleTalk.

nathanmcrae 3y ago
What packages would you recommend for this kind of usage?
- bustermellotron 3y ago
  This covers most of what you need for "housekeeping" scripts in Python: https://automatetheboringstuff.com https://automatetheboringstuff.com

BoardsOfCanada 3y ago

Maybe this already exists, but I imagine if someone wrote at library for interacting with gnu or posix commands from python (or ruby, or...) that built up the commands in a structured way and parsed the replies and put them in a dict, that would be very nice to use instead of bash-scripts. Every weird thing that you wanted to do would be a lot easier to do in python than in bash, and none of that looking out for spaces in strings and so on.

meta-meta 3y ago
https://github.com/babashka/babashka https://github.com/babashka/babashka comes to mind
throwaway858 3y ago
There is this for Haskell, which automatically converts all shell commands (like "ls") to regular Haskell functions: https://chrisdone.com/posts/shell-conduit/ https://chrisdone.com/posts/shell-conduit/ It uses a Haskell streaming library so you can do stuff like shell pipelines and file redirection (but in a more structured, safer and more powerful way)
Too 3y ago
jc is the most comprehensive I’m aware of. https://kellyjonbrazil.github.io/jc/ https://kellyjonbrazil.github.io/jc/ Except a lot of the times you don’t need this. The standard library may have what you need. There is no reason to pipe ls through jc, when you could use os.listdir().

faeriechangling 3y ago

The thing that makes me loathe Bash more than anything else is it’s 3 types are string, int, and list and those are not the only types of data I work with day to day.

vram22 3y ago

IMHO, the best resource to learn about both Unix shell usage at the command-line prompt, and shell programming in .sh script files, is still the book "The Unix Programming Environment" by Kernighan and Pike, except for shell gotchas and subtleties which were discovered later. Of course, they still would not have covered all possible issues known even at that time, because that was not the focus of the book, which was to be a Unix tutorial, not just on shell, but many other commands, general Unix usage, and the environment too (hence the title of the book). But they did cover and warn about some issues, including giving solutions.

SomeCallMeTim 3y ago

Bash/Sh is an objectively awful programming language. You could likely fill an entire book with all of the design fails that is Bash script syntax. Only in a shell script would you ever need eight backslash characters in a row in order to accomplish something...and "space as delimiter" is awful in so many ways. But especially Sh runs everywhere, so it's good to know. This seems to be the article's point. Honestly, though? The right answer is to transpile a better language to Sh. Looking around, there have been several half-hearted, abandoned attempts. But apparently everyone is happy to just fall back on the "accidental syntax" of Bash that really doesn't make sense when compared to ... any other language. I put up with Bash because there's no better alternative. That doesn't mean we should put up with Bash; just that we must put up with it at least to the point where it can run code written in a real language.

kagevf 3y ago
> "space as delimiter" is awful in so many ways Lisp and Scheme would like to have a word . . .
- deleted 3y ago
  [deleted]
- SomeCallMeTim 3y ago
  I needed to learn Lisp in college. I hate Lisp. 'nough said.
danielvaughn 3y ago
I'm not very knowledgeable in this area as I don't do much shell scripting, but isn't zsh supposed to be some kind of enhancement over bash? Does it suffer from the same issues? Regardless, the idea of a language that transpiles to bash is kinda interesting.
- NERD_ALERT 3y ago
  ZSH is basically the same thing. For all intents and purposes it is basically the same thing. The best attempt I’ve seen to truly make a better shell is Nushell. https://www.nushell.sh/ https://www.nushell.sh/

jquast 3y ago

We should have better local shells that “drive” remote sh/bash shells through transpilation, so that we can have modern shells anywhere without concern for remote install, access to install or compile, compatibility, carrying customizations, etc.

SPBS 3y ago

So it's both ubiquitous and a terse DSL for managing and coordinating programs. Yeah I agree. Also if you're on Windows it's nice that PowerShell is always preinstalled and serves the same purpose.

stanleydrew 3y ago

This argument isn't very compelling. Shell scripts are great at tricking you into believing you've got something 100% working when in fact you've only accounted for the happy path and maybe two or three failure cases. This is the primary reason shell scripts are so often broken. There are tons of assumptions baked in which are easily violated. If your shell script keeps breaking, and you keep fixing it by writing more shell, then you're in an abusive relationship with your programming environment.

nickm12 3y ago
"Shell scripts are great at tricking you into believing you've got something 100% working" Exactly. Shell scripting is basically a non-hygenic macro language, with very complex semantics around its primary data type, the string. It also lacks pretty much all of the affordances we use to write reliable programs. I have no doubt someone has written a typechecker or unit test framework for bash, but I've never seen one used. I'll take the ugly code with the straightforward semantics, thanks.

shmerl 3y ago

Good points. But I wish Bash had better syntax and built in support for basic concepts like boolean logic. Try to compose some complex boolean expression in Bash and you will quickly hit problems.

sigmonsays 3y ago

it seems like most people that have dislike bash never spent the time to learn it. I have a love/hate relationship with is. I started with computers in 2000 and wrote a ton of bash for various things. Compiling software, running cicd, adhoc batch jobs, etc. I've managed many cicd pipelines with bash in my time. While it's not great, once you know the basics, you can be very successful. That being said, rewriting the bash "program" in another language is much advised. If your bash is over a few hundred lines, chances are you've outgrown it Some things in bash (mainly pipes and redirection) are just too easy. Job control is also great. The 'process' and 'shell script' are two primitives that make a lot possible. Trying to do pipes or redirection in python is awful. There are a ton of subtle bugs you have to worry about

bluepizza 3y ago
Seems like you spent the time to learn it and still dislike it?
- sigmonsays 3y ago
  hope it didn't seem like I dislike it. It's just a tool for a job. I still write bash and love it for the simple things. Now I have the experience to rewrite it early before it gets out of hand.

makz 3y ago

I have a counterpoint for many comments here. If you are writing some Python for example, and most of your code is executing commands such as tail, cat, wc, etc. and then doing something with their output, you are better off writing a shell script instead .

DerekL 3y ago
But if you're in Python, why would you need to need to execute separate processes to do any of that?
- makz 3y ago
  I don’t know, ask the people writing that kind of stuff.
  - Leo_Germond 3y ago
    In my own experience people typically do that when they have some experience in bash, none in python, and try to do something in python the way they would do it in bash. So for those my advice, instead of getting back again to bash is to get a bit curious about the python standard lib, which has a nice doc unlinke bash, and not assume they know everything about scripting just because they have spent a long amount of time writing bash script.
    - rascul 3y ago
      > which has a nice doc unlinke bash I find the bash reference manual to be nice enough for me. https://www.gnu.org/savannah-checkouts/gnu/bash/manual/html_node/index.html https://www.gnu.org/savannah-checkouts/gnu/bash/manual/html_...

jononomo 3y ago

I have found that GPT-4 can write bash scripts for me pretty well.

andrewl 3y ago

The article recommends Bite Size Bash by Julia Evans. Does anybody have an experience with it?

asicsp 3y ago
You could check out a sample of her comics here: https://wizardzines.com/comics/ https://wizardzines.com/comics/ (there's a search box too). Her articles often hit HN front page: https://news.ycombinator.com/from?site=jvns.ca https://news.ycombinator.com/from?site=jvns.ca

owenpalmer 3y ago

1. Someone creates a platform. 2. They make a terrible language the most convenient language. 3. People use the platform, along with the terrible language. 4. This terrible language is now everywhere. 5. The argument for using this terrible language is #4

gtsop 3y ago

As a serious programmer, I have printed out and read most of the bash v5 manual. As a serious programmer, i do TDD even with bash, it is called bats-core As a serious programer I use the right tool for the right job, so I write bash scripts when it makes sense. Having done all these, I rearly write bash programs, but when I do, it is a joy! Plus i have gteater terminal understanding which is useful beyond bash programming.

amadeuspagel 3y ago

> This is a contrived example Indeed, in JS you wouldn't use awk and sort, you'd use JS array functions. It's fine to have examples that are contrived in the sense that the problem is contrived, but the solution should not be.

leke 3y ago

I was just thinking about whether or not there is a compile to bash language.