11 ms·

I've been sensing a stigma against shell scripting increasingly over the past decade. It's nice to see bash being used in creative and elaborate ways like this

I've been sensing a stigma against shell scripting increasingly over the past decade. It's nice to see bash being used in creative and elaborate ways like this.

AceJohnny2 5y ago
The stigma is well deserved and I will enthusiastically spread it. Bash should be considered an esoteric programming language. Shell Programming Considered Harmful. It's hard to program, and easy to make mistakes in. It should be left as a tool of last resort. (that said, I understand the appeal of doing weird things with it, like the Minecraft server, for the same reason other esoteric programming languages have appeal)
- asdff 5y ago
  What is so hard about it? It feels like python but with an even more english language like syntax, and the man files can be like tomes of knowledge. 'head file.txt' is a heck of a lot easier to wrap your own head around than reading something into an object and calling head on it. Python programmers today would probably import pandas for that.
  - scubbo 5y ago
    Not the person you replied to, but - yes, `head file.txt` is perfectly reasonable, as are a few other use-cases like `grep`/`sed`-ing. But when you go beyond "processing streams of lines of text" and start having to introduce conditionals, maps, or functions, the syntax is (for me, at least) opaque and easy to make mistakes in.
    - marginalia_nu 5y ago
      Isn't that also a case against learning new programming languages?
      - vore 5y ago
        I think bash has a lot of gotchas due to its original purpose, and it's definitely more error-prone to write bash scripts than to e.g. write Python scripts, so that should be a consideration. Ideally we should all try to more programming languages that are harder for us to shoot ourselves in the foot with, rather than easier ;-)
        marginalia_nu 5y ago
        I think considering the purpose of the language, it does rather well. It's really easy to spawn and pipe processes, far more so than in almost any alternatives. Like it's so easy that it's easy to forget that this is like a 5-10 line overhead dance to launch and read from a process in most languages. The problem usually come when you attempt to use it for nontrivial programming. But that just not what it's for, any more than the difficulty in building an OS kernel is a legitimate complaint against PHP.
        vore 5y ago
        I think this is what the post you were replying to is saying.
        chasil 5y ago
        Stephen Borne famously disliked C, and much preferred Algol. This is important to know in the context of what you are saying. The Algol structures in the Borne shell are the most incongruous parts of UNIX. https://research.swtch.com/shmacro https://research.swtch.com/shmacro
        chasil 5y ago
        I will just leave this here. "I am a strong believer that Bourne-derived languages are extremely bad, on the same order of badness as Perl, for programming, and consider programming sh for any purpose other than as a super-portable, lowest-common-denominator platform for build or bootstrap scripts and the like, as an extremely misguided endeavor. As such you won’t see me spending many words on extensions particular to ksh, Bash, or whatever other shells may be popular." http://www.etalabs.net/sh_tricks.html http://www.etalabs.net/sh_tricks.html
  - pyuser583 5y ago
    I can tell you’ve dealt with lots of Python programmers.
  - _dain_ 5y ago
    >What is so hard about it? quick: without looking anything up, write something in bash that takes a string of comma-separated values and turns it into an array of those values. then check to see if your answer is on this very long list of wrong implementations: https://stackoverflow.com/a/45201229/6107981 https://stackoverflow.com/a/45201229/6107981
    - asdff 5y ago
      Easy, I just avoid these sort of mind game situations when programming, and I come up with a simple 'wrong' approach that will probably work fine for my purposes anyhow. I don't care if my implementation breaks if I have a newline in my data or whatever, because I already know based on what I am working on that I won't have to worry about a newline in my data or any other weird quirks they bring up in stack overflow comment sections that I have yet to come across in my years of writing code that gets the job done. I don't care if my solution is fractions of a second slower, it's probably already going to be mindbendingly fast compared to the code running on hardware from 40 years ago when people had to be a lot more mindful about selecting the most performant solutions when writing in bash or anything else.
      - _dain_ 5y ago
        Okay, but how do you know that those weird quirks won't be present in your data? Or what those weird quirks even are? The details of the stuff in that Stackoverflow comment have already fallen out of my head; I couldn't tell you about them now. With Bash there are so many traps and footguns and (subjective) unknown unknowns, it would keep me up at night worrying about it. There's always another maddening fact to discover. I would just use Python, where strings are sane and lists are sane: my_string.split(", "), boom, done, no jumpscares.
- rascul 5y ago
  > Bash should be considered an esoteric programming language. Shell Programming Considered Harmful. I disagree. Bash is fine. Nothing esoteric about it. > It's hard to program, and easy to make mistakes in. It should be left as a tool of last resort. It's easy to program, but it has a learning curve. Use shellcheck. It's OK to go first to bash if you need a shell script. Other languages make pretty crappy shell scripts.
  - pizza234 5y ago
    I copy/paste from a previous post. "Exoteric" is a stretch (in particular, because there are no alternative glue languages, if we exclude all the shell variants), but cryptic, unintuitive and error-prone fits well. Arrays: - `${#myvar[@]}`: unreadable - `mapfile myvar < <(grep pattern file)`: "mapfile" is not exactly a clear term (there's the synonym "readarray", but "mapfile" is the reference; worse, you may find both used inconsistently); the command as I wrote it also has two problems Strings: - `${myvar##.*}`: unreadable - `echo $(IFS=,; echo "${myvar[*]}")`: unreadable; most also don't know the difference between `[@]` and `[*]`; this doesn't also work for two-char join operations - `[[ $myvar =~ $(echo '\bpattern\b') ]]`: unreadable, and most don't know why command substitution is needed Redirections: - `3>&2 2>&1 1>&3`: most don't know what this means
    - marklgr 5y ago
      Yes, it can be cryptic and not beginner-friendly, but it's powerful and once you know it, you know it; it will be useful for years if not decades, as opposed to some tools or frameworks du jour that are sometimes as cryptic. The main issue is that many people try (by necessity or not) to cobble scripts together without really learning the language and its idioms. It's not that hard to really learn it though, it's basically one man page then the mandatory [wooledge wiki][1]. [1]: http://mywiki.wooledge.org/BashGuide http://mywiki.wooledge.org/BashGuide
    - rascul 5y ago
      I don't understand, none of that is unreadable or weird to me. Are you sure the issue is not a lack of familiarity with the language? The language can appear deceptively simple in the beginning.
- kdtsh 5y ago
  And what a tool of last resort it is. It may be much easier (and most of the time I would generally prefer) to use Python for scripting purposes, but Bash is a) almost always available and b) widely portable. If you manage any servers where you never know if Python is available or if it’s version 2 or 3 (and you have no power to change this for one reason or other), you know the value of this. No, it’s not ideal, but we live in a non-ideal world. Long live Bash.
- AtlasBarfed 5y ago
  Bash has the highest rate of Stackoverflow searches per line of code produced. I swear I do about 10 searches per line of code produced. And when you consider there really isn't a standard library, mostly gee-I-hope-this-UNIX-util-is-installed...
- francis-io 5y ago
  I think the big thing going for it is it's low footprint and mostly universal existence on all servers I work on. For example, I'm working on a Hashicorp Vault server install and due to it's sensitive nature, I'm strongly considering using bash to set it up over something like python. I love the ability to test in a robust language, but the extra dependencies are hard to justify, at least on a security sensitive application.
- einpoklum 5y ago
  > hard to program Well, we "write" one-liner programs with it all day every day - since it's the shell. I'm not saying bash doesn't have a bunch of warts, but - what other shell language would you suggest as an alternative to it?
atarian 5y ago
Yeah it gets bashed a lot.
- einpoklum 5y ago
  Yet we continue to use it unabashed.
usrbinbash 5y ago
The stigma is not against shell scripting, the stigma is against using shell scripting for things and in settings it wasn't intended for. Don't get me wrong, I have nothing against fun and creative projects. If someone wants to script something complicated in bash for the lulz, the challenge, to learn something, or just to show off bash-skills, I'm all for it. What I am against, is when I get asked to debug a 2000 LOC system used in production that was thrown together in bash, waste 2 days on it, only to end up doing what should have been done in the first place, and rewrite the damn thing in 200 lines of Python or 400 lines of Go. It can, and should, be used to automate some things, string some commands together, make certain tasks easier. I use and write bash scripts in my daily work, and make my life easier with it. But there comes a point, and that is sooner rather than later, when something should no longer be written in bash, but in a language that wasn't designed primarily as a command line interpreter. Can I make large complicated systems in bash? Sure. Can I carve the handle of a hammer so that it can be used as a screwdriver? I'm sure someone more skilled in woodworking than me could do it. But sometimes the question isn't if it can be done, but if it should.
dspillett 5y ago
There has always been stigma against shell programming, not shell scripting, and I think that more-or-less is still the case. It isn't so much anti-bash stigma as, using a less than ideal tool for the job being a smell that'll cost you to fix later. Shell scripts to automate processes with other tools is exactly what they are designed for, and adding in other logic and loops is fine too, but if you are setting out to create what would be considered a full program rather than an automated task you should definitely consider alternatives (of which there are several suitable choices) that usually end up being easier to maintain. It used to be that the main alternative was perl, but frankly that was/is no better (IMO much worse) than bash for encouraging the generation write-only (difficult to maintain) code. Python is often a good option, and I've seen node used, though these don't have the ubiquity that sh/bash does (or perl used to have) which is one advantage of bash: it is almost everywhere, even Windows these days (or stick with core posix sh and you'll find support a little wider again), though maybe without some of the supporting externals which is when tricks like these come in. If you are writing some simple automation, or maybe slightly less simple, then bash is fine as a go-to, but if you are setting out to write something complex bash is a bad choice unless compatibility with many environments you can't dictate have another interpreter present is a non-negotiable requirement. There is the grey area when a script, or collection of scripts, becomes a larger program/environment over time. Sometimes the effort to rewrite (and fully test) in another environment is not worth the small benefit of doing so, and is therefore a bad use of developer/admin time that could be used elsewhere, particularly when high performance is not an important feature. Though if you find yourself with a lot of bash script, do yourself (and your later replacement when you've moved on to newer projects) a massive favour and make sure it is well documented even if that is only in in-script comments.

I've been sensing a stigma against shell scripting increasingly over the past decade. It's nice to see bash being used in creative and elaborate ways like this.

AceJohnny2 5y ago

The stigma is well deserved and I will enthusiastically spread it. Bash should be considered an esoteric programming language. Shell Programming Considered Harmful. It's hard to program, and easy to make mistakes in. It should be left as a tool of last resort. (that said, I understand the appeal of doing weird things with it, like the Minecraft server, for the same reason other esoteric programming languages have appeal)

asdff 5y ago
What is so hard about it? It feels like python but with an even more english language like syntax, and the man files can be like tomes of knowledge. 'head file.txt' is a heck of a lot easier to wrap your own head around than reading something into an object and calling head on it. Python programmers today would probably import pandas for that.
- scubbo 5y ago
  Not the person you replied to, but - yes, `head file.txt` is perfectly reasonable, as are a few other use-cases like `grep`/`sed`-ing. But when you go beyond "processing streams of lines of text" and start having to introduce conditionals, maps, or functions, the syntax is (for me, at least) opaque and easy to make mistakes in.
  - marginalia_nu 5y ago
    Isn't that also a case against learning new programming languages?
    - vore 5y ago
      I think bash has a lot of gotchas due to its original purpose, and it's definitely more error-prone to write bash scripts than to e.g. write Python scripts, so that should be a consideration. Ideally we should all try to more programming languages that are harder for us to shoot ourselves in the foot with, rather than easier ;-)
      - marginalia_nu 5y ago
        I think considering the purpose of the language, it does rather well. It's really easy to spawn and pipe processes, far more so than in almost any alternatives. Like it's so easy that it's easy to forget that this is like a 5-10 line overhead dance to launch and read from a process in most languages. The problem usually come when you attempt to use it for nontrivial programming. But that just not what it's for, any more than the difficulty in building an OS kernel is a legitimate complaint against PHP.
        vore 5y ago
        I think this is what the post you were replying to is saying.
        chasil 5y ago
        Stephen Borne famously disliked C, and much preferred Algol. This is important to know in the context of what you are saying. The Algol structures in the Borne shell are the most incongruous parts of UNIX. https://research.swtch.com/shmacro https://research.swtch.com/shmacro
      - chasil 5y ago
        I will just leave this here. "I am a strong believer that Bourne-derived languages are extremely bad, on the same order of badness as Perl, for programming, and consider programming sh for any purpose other than as a super-portable, lowest-common-denominator platform for build or bootstrap scripts and the like, as an extremely misguided endeavor. As such you won’t see me spending many words on extensions particular to ksh, Bash, or whatever other shells may be popular." http://www.etalabs.net/sh_tricks.html http://www.etalabs.net/sh_tricks.html
- pyuser583 5y ago
  I can tell you’ve dealt with lots of Python programmers.
- _dain_ 5y ago
  >What is so hard about it? quick: without looking anything up, write something in bash that takes a string of comma-separated values and turns it into an array of those values. then check to see if your answer is on this very long list of wrong implementations: https://stackoverflow.com/a/45201229/6107981 https://stackoverflow.com/a/45201229/6107981
  - asdff 5y ago
    Easy, I just avoid these sort of mind game situations when programming, and I come up with a simple 'wrong' approach that will probably work fine for my purposes anyhow. I don't care if my implementation breaks if I have a newline in my data or whatever, because I already know based on what I am working on that I won't have to worry about a newline in my data or any other weird quirks they bring up in stack overflow comment sections that I have yet to come across in my years of writing code that gets the job done. I don't care if my solution is fractions of a second slower, it's probably already going to be mindbendingly fast compared to the code running on hardware from 40 years ago when people had to be a lot more mindful about selecting the most performant solutions when writing in bash or anything else.
    - _dain_ 5y ago
      Okay, but how do you know that those weird quirks won't be present in your data? Or what those weird quirks even are? The details of the stuff in that Stackoverflow comment have already fallen out of my head; I couldn't tell you about them now. With Bash there are so many traps and footguns and (subjective) unknown unknowns, it would keep me up at night worrying about it. There's always another maddening fact to discover. I would just use Python, where strings are sane and lists are sane: my_string.split(", "), boom, done, no jumpscares.
rascul 5y ago
> Bash should be considered an esoteric programming language. Shell Programming Considered Harmful. I disagree. Bash is fine. Nothing esoteric about it. > It's hard to program, and easy to make mistakes in. It should be left as a tool of last resort. It's easy to program, but it has a learning curve. Use shellcheck. It's OK to go first to bash if you need a shell script. Other languages make pretty crappy shell scripts.
- pizza234 5y ago
  I copy/paste from a previous post. "Exoteric" is a stretch (in particular, because there are no alternative glue languages, if we exclude all the shell variants), but cryptic, unintuitive and error-prone fits well. Arrays: - `${#myvar[@]}`: unreadable - `mapfile myvar < <(grep pattern file)`: "mapfile" is not exactly a clear term (there's the synonym "readarray", but "mapfile" is the reference; worse, you may find both used inconsistently); the command as I wrote it also has two problems Strings: - `${myvar##.*}`: unreadable - `echo $(IFS=,; echo "${myvar[*]}")`: unreadable; most also don't know the difference between `[@]` and `[*]`; this doesn't also work for two-char join operations - `[[ $myvar =~ $(echo '\bpattern\b') ]]`: unreadable, and most don't know why command substitution is needed Redirections: - `3>&2 2>&1 1>&3`: most don't know what this means
  - marklgr 5y ago
    Yes, it can be cryptic and not beginner-friendly, but it's powerful and once you know it, you know it; it will be useful for years if not decades, as opposed to some tools or frameworks du jour that are sometimes as cryptic. The main issue is that many people try (by necessity or not) to cobble scripts together without really learning the language and its idioms. It's not that hard to really learn it though, it's basically one man page then the mandatory [wooledge wiki][1]. [1]: http://mywiki.wooledge.org/BashGuide http://mywiki.wooledge.org/BashGuide
  - rascul 5y ago
    I don't understand, none of that is unreadable or weird to me. Are you sure the issue is not a lack of familiarity with the language? The language can appear deceptively simple in the beginning.
kdtsh 5y ago
And what a tool of last resort it is. It may be much easier (and most of the time I would generally prefer) to use Python for scripting purposes, but Bash is a) almost always available and b) widely portable. If you manage any servers where you never know if Python is available or if it’s version 2 or 3 (and you have no power to change this for one reason or other), you know the value of this. No, it’s not ideal, but we live in a non-ideal world. Long live Bash.
AtlasBarfed 5y ago
Bash has the highest rate of Stackoverflow searches per line of code produced. I swear I do about 10 searches per line of code produced. And when you consider there really isn't a standard library, mostly gee-I-hope-this-UNIX-util-is-installed...
francis-io 5y ago
I think the big thing going for it is it's low footprint and mostly universal existence on all servers I work on. For example, I'm working on a Hashicorp Vault server install and due to it's sensitive nature, I'm strongly considering using bash to set it up over something like python. I love the ability to test in a robust language, but the extra dependencies are hard to justify, at least on a security sensitive application.
einpoklum 5y ago
> hard to program Well, we "write" one-liner programs with it all day every day - since it's the shell. I'm not saying bash doesn't have a bunch of warts, but - what other shell language would you suggest as an alternative to it?

atarian 5y ago

Yeah it gets bashed a lot.

einpoklum 5y ago
Yet we continue to use it unabashed.

usrbinbash 5y ago

The stigma is not against shell scripting, the stigma is against using shell scripting for things and in settings it wasn't intended for. Don't get me wrong, I have nothing against fun and creative projects. If someone wants to script something complicated in bash for the lulz, the challenge, to learn something, or just to show off bash-skills, I'm all for it. What I am against, is when I get asked to debug a 2000 LOC system used in production that was thrown together in bash, waste 2 days on it, only to end up doing what should have been done in the first place, and rewrite the damn thing in 200 lines of Python or 400 lines of Go. It can, and should, be used to automate some things, string some commands together, make certain tasks easier. I use and write bash scripts in my daily work, and make my life easier with it. But there comes a point, and that is sooner rather than later, when something should no longer be written in bash, but in a language that wasn't designed primarily as a command line interpreter. Can I make large complicated systems in bash? Sure. Can I carve the handle of a hammer so that it can be used as a screwdriver? I'm sure someone more skilled in woodworking than me could do it. But sometimes the question isn't if it can be done, but if it should.

dspillett 5y ago

There has always been stigma against shell programming, not shell scripting, and I think that more-or-less is still the case. It isn't so much anti-bash stigma as, using a less than ideal tool for the job being a smell that'll cost you to fix later. Shell scripts to automate processes with other tools is exactly what they are designed for, and adding in other logic and loops is fine too, but if you are setting out to create what would be considered a full program rather than an automated task you should definitely consider alternatives (of which there are several suitable choices) that usually end up being easier to maintain. It used to be that the main alternative was perl, but frankly that was/is no better (IMO much worse) than bash for encouraging the generation write-only (difficult to maintain) code. Python is often a good option, and I've seen node used, though these don't have the ubiquity that sh/bash does (or perl used to have) which is one advantage of bash: it is almost everywhere, even Windows these days (or stick with core posix sh and you'll find support a little wider again), though maybe without some of the supporting externals which is when tricks like these come in. If you are writing some simple automation, or maybe slightly less simple, then bash is fine as a go-to, but if you are setting out to write something complex bash is a bad choice unless compatibility with many environments you can't dictate have another interpreter present is a non-negotiable requirement. There is the grey area when a script, or collection of scripts, becomes a larger program/environment over time. Sometimes the effort to rewrite (and fully test) in another environment is not worth the small benefit of doing so, and is therefore a bad use of developer/admin time that could be used elsewhere, particularly when high performance is not an important feature. Though if you find yourself with a lot of bash script, do yourself (and your later replacement when you've moved on to newer projects) a massive favour and make sure it is well documented even if that is only in in-script comments.