4 ms·

Different tools are better at different things. Shell is required to be basically everywhere. It's part of POSIX. It's also standardized. Shell scripts written

Different tools are better at different things. Shell is required to be basically everywhere. It's part of POSIX. It's also standardized. Shell scripts written 30 years ago still work, and will probably work 30 years from now too. Shell is probably on your TV. Not everyone has Python. The most common version in the field is Python2, but that's officially obsolete. Python3 is in many places, but not everywhere. The two Pythons are basically incompatible unless you're willing to put in an effort. You have to work to make Python scripts compatible with both versions, especially if you can only use built-ins, and the results are way clunkier than that 2-line shell script you're trying to replace. Python is a terrible choice if your goal is a relatively short script that calls other programs. You can call out to other programs in Python, but it's much clunkier and more painful to maintain. And while it often doesn't matter (because neither are speedy), CPython 3 takes more than 50x more time to start than dash and 27x time to start than bash (see: https://github.com/bdrung/startup-time https://github.com/bdrung/startup-time , which shows Python3 3.6.4 at 197.79 ms, Bash 4.4.12 at 7.31 ms, and dash 0.5.8 at 2.81 ms). That speed is helpful when you want to have a "simple script in a loop". Python is much better than shell when you start needing more sophisticated data types, libraries, modularity, etc. But the typical use for a shell script doesn't need any of that. If you need that, shell is a terrible tool, because it's the wrong tool for the job, not because it's never useful. If you're writing shell, use shellcheck. Once you do that, your error likelihood goes way down. Many of the reports of problems writing shell come from a time when shellcheck didn't exist.

nh2 7y ago
These are fair points. POSIX shell being fixed forever though also means it will /never/ get "fixed". Shell's silent-failure-by-default, implicit-by-default, principle-of-most-surprise semantics will always be there, and they are /bad/. I have read lots of shell scripts, by many people, from the experienced UNIX greybeard to to the novice web developer, and they were almost all flawed. Pointing out the flaws, the devlopers of each level were surprised by them. It is close to impossible to write correct shell script, no matter the level of expertise, because the language offers neither semantics nor tooling that foster correctness. Forgetting `set -e`, forgetting that it gets turned back off in subshells, quoting mistakes, pipefail semantics, you have to have made many mistakes to even /discover/ that these pitfalls exist (or use shellcheck, which will find some, but certainly not all of them). Shell is a language that looks simple on the surface, but has 1000s of exceptions you have to master to write even minimal non-bugged scripts. The lack of any form of reasonable data structures and strings being the prevalent data types also do not help. For example, NixOS's linker wrapper script (written in bash) was accidentally quadratic because of quadratic append-to-strings (https://github.com/NixOS/nixpkgs/issues/27609); https://github.com/NixOS/nixpkgs/issues/27609); and that was despite NixOS having many competent bash programmers. When scripts grow, these things happen.(Note that shellcheck does /not/ point out quadratic string appends; it can only find obvious mistakes and this is hard to detect in general). Fixing it was very difficult, and was not understood by many. If you have a language like Python that offers reasonable set/dict data structures that people understand and use intuitively, this is trivially avoided. Finally, POSIX does not help much when the main thing shell does is calling other programs, and there are no guarantees on what the APIs for those programs are. For example, I have encountered many shell scripts that themselves are technically "portable", but they use arguments to e.g. `rm` that do not work on e.g. busybox, thus crashing the boot processe and resulting in hours of time lost for hundreds of people. By not using shell, we can support that the next SOMETHING-standard will NOT mandate shell, but something better.
- im3w1l 7y ago
  Compared to a proper language, shellscripts work for 80% of inputs with 20% of the code.
filesystemdude 7y ago
This. A Python script is not the answer when I'm trying to do something quick and dirty at the shell. The beauty of shell scripting is that it evolves seamlessly from trying to solve simple problems at my command prompt. I pipe a couple of things together, and then I realize I could use a loop and a few conditionals, and suddenly it makes sense to store this in file in case I want to do this again. Boom, program done. It now works on every computer I own. I love Python. I use it almost every day. But I'll be damned if I'm going to go rewrite every shell snippet I hack together into Python just because it exists. Different problems, different solutions.
- mturmon 7y ago
  Your observation about the seamless evolution of a basic shell script from a few lines typed interactively is very insightful. This kind of automation adds a lot of value for low marginal effort and probably explains a lot of the short scripts I have laying around in directories.
- unlinked_dll 7y ago
  True that. But everyone should keep in mind that script bloat is real. If you're not careful you'll end up like me, and accidentally have to maintain what is essentially npm but written entirely in bash.

nh2 7y ago

These are fair points. POSIX shell being fixed forever though also means it will /never/ get "fixed". Shell's silent-failure-by-default, implicit-by-default, principle-of-most-surprise semantics will always be there, and they are /bad/. I have read lots of shell scripts, by many people, from the experienced UNIX greybeard to to the novice web developer, and they were almost all flawed. Pointing out the flaws, the devlopers of each level were surprised by them. It is close to impossible to write correct shell script, no matter the level of expertise, because the language offers neither semantics nor tooling that foster correctness. Forgetting `set -e`, forgetting that it gets turned back off in subshells, quoting mistakes, pipefail semantics, you have to have made many mistakes to even /discover/ that these pitfalls exist (or use shellcheck, which will find some, but certainly not all of them). Shell is a language that looks simple on the surface, but has 1000s of exceptions you have to master to write even minimal non-bugged scripts. The lack of any form of reasonable data structures and strings being the prevalent data types also do not help. For example, NixOS's linker wrapper script (written in bash) was accidentally quadratic because of quadratic append-to-strings (https://github.com/NixOS/nixpkgs/issues/27609); https://github.com/NixOS/nixpkgs/issues/27609); and that was despite NixOS having many competent bash programmers. When scripts grow, these things happen.(Note that shellcheck does /not/ point out quadratic string appends; it can only find obvious mistakes and this is hard to detect in general). Fixing it was very difficult, and was not understood by many. If you have a language like Python that offers reasonable set/dict data structures that people understand and use intuitively, this is trivially avoided. Finally, POSIX does not help much when the main thing shell does is calling other programs, and there are no guarantees on what the APIs for those programs are. For example, I have encountered many shell scripts that themselves are technically "portable", but they use arguments to e.g. `rm` that do not work on e.g. busybox, thus crashing the boot processe and resulting in hours of time lost for hundreds of people. By not using shell, we can support that the next SOMETHING-standard will NOT mandate shell, but something better.

im3w1l 7y ago
Compared to a proper language, shellscripts work for 80% of inputs with 20% of the code.

filesystemdude 7y ago

This. A Python script is not the answer when I'm trying to do something quick and dirty at the shell. The beauty of shell scripting is that it evolves seamlessly from trying to solve simple problems at my command prompt. I pipe a couple of things together, and then I realize I could use a loop and a few conditionals, and suddenly it makes sense to store this in file in case I want to do this again. Boom, program done. It now works on every computer I own. I love Python. I use it almost every day. But I'll be damned if I'm going to go rewrite every shell snippet I hack together into Python just because it exists. Different problems, different solutions.

mturmon 7y ago
Your observation about the seamless evolution of a basic shell script from a few lines typed interactively is very insightful. This kind of automation adds a lot of value for low marginal effort and probably explains a lot of the short scripts I have laying around in directories.
unlinked_dll 7y ago
True that. But everyone should keep in mind that script bloat is real. If you're not careful you'll end up like me, and accidentally have to maintain what is essentially npm but written entirely in bash.