3 ms·

Whenever people look for "better shell scripting", they always run into the second-system effect. Trying to shove everything that wasn't in the first system, in

Whenever people look for "better shell scripting", they always run into the second-system effect. Trying to shove everything that wasn't in the first system, into the second system. Bash shell scripting is already close to perfect for what it is intended. Where people always complain is one of three areas: 1) "footguns", 2) "not enough features", 3) "not a programming language", 4) inexperience. 1) is almost entirely due to it being a programming-like language bolted onto an interactive console terminal. Bash has a lot of changes over vanilla Bourne/Posix that make it less console-y and more programming-y. But if you take away the interactive-terminal-ness itself, then you don't need most of the Bashisms. 2) is almost always a mistake. Bash has pretty much all the features you need. But again, it's hampered by the interactive-terminal-ness; take that away, and you can do pretty much anything, with ease. 3) is because programmers don't understand what scripts are for. They are not for writing robust programs. They are for solving a small problem, in a specific, tailored way, with the least amount of effort and time. This means that you should be abandoning all of the programmer's best practices and rules. Throw out safety, reliability, consistency, verifiability, etc. What you end up with is a solution that works, with nothing extra. 4) comes when people simply don't know how to use Bash very well, which is 90% of the time. Ever since I became a programmer two decades ago, the great unwashed masses of techno-dweebs who call themselves "programmers" have been picking up complex and nuanced technology that they have never bother to formally learn, and then complain that it sucks and they need an alternative. When in reality, the problem is almost always either a) they didn't read the manual, b) they haven't practiced using it, or c) they're using the wrong tool anyway. Doesn't matter if you disagree with me. We will be using the same shell scripts in 20 years. Know why? Because you can't make it any simpler, and it just works. Complain all you want about it not being perfect. You simply will not be able to make something better without simultaneously making it worse. Its killer feature is its lack of features and design. It is the anti-programming language. And boy do I love it.

shlomo_z 2y ago
well said!
ryandv 2y ago
> Ever since I became a programmer two decades ago, the great unwashed masses of techno-dweebs who call themselves "programmers" have been picking up complex and nuanced technology that they have never bother to formally learn, and then complain that it sucks and they need an alternative. When in reality, the problem is almost always either a) they didn't read the manual, b) they haven't practiced using it This kind of Dunning-Kruger-esque arrogance in software is intensely aggravating to me. Nobody presumes that you can master, say, a musical instrument, without intensive practice over a timespan of years; yet for some reason people can get plopped out the other end of a bootcamp and suddenly identify as world class programmers in 16 weeks without the requisite thousands of hours of practice with hands on keyboard, reading the manual, studying how the computer works, and actually solving technical problems.
- bigiain 2y ago
  I've been around a while. As in I've installed Linux from floppies kind of vintage. I get new grads with great grades, who are decent junior devs that are productive in, say, Java and one or maybe two related things like SpringBoot or Hikari, but don't even know about simple/common things that sometimes baffle me. They seem to think I'm some sort of wizard because I can quickly put together simple commands line pipelines using things like grep, awk, sed, sort, uniq, wc - and dig useful answers or insights out of log files or csv files or database exports, or even JSON by using jq as well. It amazes me that some of them have never even heard of the existence of tools like that, or how easily you can pipe the output from one to the input of another and chain them up in such useful ways. (At the same time, I'm fairly sure I have similar black holes in my knowledge and mental map of how things that other people specialise in work. :shrug: )
  - BeetleB 2y ago
    > It amazes me that some of them have never even heard of the existence of tools like that I'm not a recent grad by any means. I have heard of those tools and even used them on occasion. The problem is "on occasion". There's no way I want to keep relearning awk every time it might be useful. These days, though, LLMs are a saving grace. It's rare that I ask it to provide me with a command line to solve a problem and it gets it wrong. I still have the burden of verifying, but it's a happy medium between your expertise and my writing everything in Python.
bigiain 2y ago
#4 is the big one for me. And I don't really care. I'm super comfortable with one-liners in my bash shell. I can do useful stuff in bash scripts, but typically with a base script gets to more than a dozen or so lines, I find myself way more productive rewriting it in Python instead. Not because Python is "better" or because bash is "worse", but because _I'm_ better with Python and _I'm_ worse with bash. And I don't feel any urgent need to get better at bash, because the work I do means I pretty much always have Python available (at least everywhere I have bash available). I know bash is way more capable than my kindergarten-grade use of it. And it'll remain in my toolkit and crontab for as long as unix-like OSen exist. But I'll be very unlikely to ever keep writing a bash script past a dozen or two lines ever again.
- BeetleB 2y ago
  Try using xonsh: https://xon.sh/ https://xon.sh/ I write all my shell scripts in it (well, at least the ones I don't need to share with coworkers).
chasil 2y ago
"Bash shell scripting is already close to perfect for what it is intended." It is very difficult to write a parser for any derivative of the POSIX shell. The lex/yacc utilities cannot be used; the lexer and parser must exchange state. It would be better if the shell was an LR-parsed language with a yacc grammar. The yacc utility is itself in POSIX.2. Here is a talk about an OCaml implementation of a parser for the POSIX shell: https://archive.fosdem.org/2018/schedule/event/code_parsing_posix_s_hell/ https://archive.fosdem.org/2018/schedule/event/code_parsing_... Here is another talk about an Ada parser implementation: https://archive.fosdem.org/2019/schedule/event/ada_shell/ https://archive.fosdem.org/2019/schedule/event/ada_shell/ The language grammar is unfortunate.
Izkata 2y ago
On 4, my favorite example is hearing "do I quote this?" when trying to explain something. It's such a simple question, but quotes don't delimit strings like they do in other programming languages, they do something else entirely, so getting into it really reveals they have almost no clue how the language works.
BeetleB 2y ago
Can you explain to us how (beyond ubiquity and POSIX compliance) bash scripting is superior to the following: 1. fish 2. xonsh 3. nushell I often find people making such claims simply don't know much about other shells.

shlomo_z 2y ago

well said!

ryandv 2y ago

> Ever since I became a programmer two decades ago, the great unwashed masses of techno-dweebs who call themselves "programmers" have been picking up complex and nuanced technology that they have never bother to formally learn, and then complain that it sucks and they need an alternative. When in reality, the problem is almost always either a) they didn't read the manual, b) they haven't practiced using it This kind of Dunning-Kruger-esque arrogance in software is intensely aggravating to me. Nobody presumes that you can master, say, a musical instrument, without intensive practice over a timespan of years; yet for some reason people can get plopped out the other end of a bootcamp and suddenly identify as world class programmers in 16 weeks without the requisite thousands of hours of practice with hands on keyboard, reading the manual, studying how the computer works, and actually solving technical problems.

bigiain 2y ago
I've been around a while. As in I've installed Linux from floppies kind of vintage. I get new grads with great grades, who are decent junior devs that are productive in, say, Java and one or maybe two related things like SpringBoot or Hikari, but don't even know about simple/common things that sometimes baffle me. They seem to think I'm some sort of wizard because I can quickly put together simple commands line pipelines using things like grep, awk, sed, sort, uniq, wc - and dig useful answers or insights out of log files or csv files or database exports, or even JSON by using jq as well. It amazes me that some of them have never even heard of the existence of tools like that, or how easily you can pipe the output from one to the input of another and chain them up in such useful ways. (At the same time, I'm fairly sure I have similar black holes in my knowledge and mental map of how things that other people specialise in work. :shrug: )
- BeetleB 2y ago
  > It amazes me that some of them have never even heard of the existence of tools like that I'm not a recent grad by any means. I have heard of those tools and even used them on occasion. The problem is "on occasion". There's no way I want to keep relearning awk every time it might be useful. These days, though, LLMs are a saving grace. It's rare that I ask it to provide me with a command line to solve a problem and it gets it wrong. I still have the burden of verifying, but it's a happy medium between your expertise and my writing everything in Python.

bigiain 2y ago

#4 is the big one for me. And I don't really care. I'm super comfortable with one-liners in my bash shell. I can do useful stuff in bash scripts, but typically with a base script gets to more than a dozen or so lines, I find myself way more productive rewriting it in Python instead. Not because Python is "better" or because bash is "worse", but because _I'm_ better with Python and _I'm_ worse with bash. And I don't feel any urgent need to get better at bash, because the work I do means I pretty much always have Python available (at least everywhere I have bash available). I know bash is way more capable than my kindergarten-grade use of it. And it'll remain in my toolkit and crontab for as long as unix-like OSen exist. But I'll be very unlikely to ever keep writing a bash script past a dozen or two lines ever again.

BeetleB 2y ago
Try using xonsh: https://xon.sh/ https://xon.sh/ I write all my shell scripts in it (well, at least the ones I don't need to share with coworkers).

chasil 2y ago

"Bash shell scripting is already close to perfect for what it is intended." It is very difficult to write a parser for any derivative of the POSIX shell. The lex/yacc utilities cannot be used; the lexer and parser must exchange state. It would be better if the shell was an LR-parsed language with a yacc grammar. The yacc utility is itself in POSIX.2. Here is a talk about an OCaml implementation of a parser for the POSIX shell: https://archive.fosdem.org/2018/schedule/event/code_parsing_posix_s_hell/ https://archive.fosdem.org/2018/schedule/event/code_parsing_... Here is another talk about an Ada parser implementation: https://archive.fosdem.org/2019/schedule/event/ada_shell/ https://archive.fosdem.org/2019/schedule/event/ada_shell/ The language grammar is unfortunate.

Izkata 2y ago

On 4, my favorite example is hearing "do I quote this?" when trying to explain something. It's such a simple question, but quotes don't delimit strings like they do in other programming languages, they do something else entirely, so getting into it really reveals they have almost no clue how the language works.

BeetleB 2y ago

Can you explain to us how (beyond ubiquity and POSIX compliance) bash scripting is superior to the following: 1. fish 2. xonsh 3. nushell I often find people making such claims simply don't know much about other shells.