7 ms·

Sometimes you write POSIX shell (or very nearly POSIX) for portability. Sometimes you write in "a proper programming language" for more powerful/flexible featur

Sometimes you write POSIX shell (or very nearly POSIX) for portability. Sometimes you write in "a proper programming language" for more powerful/flexible features. And sometimes you write Bash to get both. Bash is ported to basically every system, it's a small binary (by today's grotesque standards), and it gives you some added functionality that would be otherwise annoying to re-implement in POSIX sh (or Bourne sh). But it's also far less complex than Python. In today's world, if someone has any shell, it's a safe bet that they either already have Bash, or they can install it just like they'd have to install Python. It's less painful in the long term to use Bash rather than Python. Python is more time-consuming and costly. It's a larger space requirement, you have to manage more runtime dependencies and a virtual environment, debugging is worse, and there's more opportunity for bugs because it's more complex. Bash scripts also rarely get bugs down the line due to dependencies changing, whereas this happens in Python so frequently we have to pin versions in a virtualenv. When's the last time you pinned the version of a Bash script dependency?

lordgroff 6y ago
This used to be my take, but I'm not sure any more. Now, I don't think Python is ideal, and I am looking for something that maybe fits better than it in that gray zone of programming language vs "quick" script, but I can't see your point about debugging and bugs: bash is notoriously annoying to write bug free, and I can't see what's the issue with the Python debugger. Also if we're in that gray zone of bash vs Python, you can pretty much stick to the stdlib.
- dylan604 6y ago
  python3 or python2.7???
  - brobinson 6y ago
    I would go with the one that hasn't been out of support since Jan 1, 2020. https://www.python.org/doc/sunset-python-2/ https://www.python.org/doc/sunset-python-2/
    - dylan604 6y ago
      yet everything comes standard with stock install of 2.7. ???
      - chungy 6y ago
        No responsible distro still ships 2.7
        dylan604 6y ago
        hahahaha. I like your definition or responsible as AWS Linux 2 comes with 2.7 and macOS comes with 2.7 glad to know i'm such a hellion running such irresponsible OSes! down with the man!! I do what I want! Or more likely, the world isn't as clear cut as you might think
        chungy 6y ago
        > I['d] like your definition [f]or responsible A distribution that removes Python 2.7 once upstream security support has been cut. Since Python is frequently used on network-facing services, it is unconscionable to allow its use past January 2020. Heck, one might even have setuid programs in Python and you don't really want to risk an unfixed local privilege escalation either. > AWS Linux 2 A butchered version of CentOS > and macOS Not a distribution, and Apple doesn't exactly have a good track record with keeping things up to date or secure. > Or more likely, the world isn't as clear cut as you might think Given that the vast majority of the Python ecosystem is no longer compatible with 2.7, there isn't even a technical reason for kicking around 2.7 anymore. Python 2.7 is trivial to build and run if you really, really want or need it, but since all the major upstream distributions have removed it, it sends a clear signal that you must take the risks into serious consideration before doing it.
- scoopdewoop 6y ago
  I'm hopeful that Deno can be just that runtime. Since its a single binary, its trivial to install and script for. TBH, I don't have any technical reasons for JS/TS over python besides that I have coworkers that have never had to learn python and would prefer not to. /** * cat.ts */ for (let i = 0; i < Deno.args.length; i++) { const filename = Deno.args[i]; const file = await Deno.open(filename); await Deno.copy(file, Deno.stdout); file.close(); }
- fomine3 6y ago
  Ruby!
gmfawcett 6y ago
In the space of shell programming, virtualenv and third-party Python libraries are irrelevant and unnecessary.
- rattray 6y ago
  Huh? That's only true if you don't use any libraries, right? What if you want to use something like delegator[0] to make running shell commands less toilsome? [0] https://github.com/amitt001/delegator.py https://github.com/amitt001/delegator.py
  - lmm 6y ago
    Python with no libraries is still a much, much nicer programming environment than bash.
    - dylan604 6y ago
      How say you? To even ask the system a question in python, you have to "import os". Is that not a library? I'm not a python-first type of person, but I have hacked enough scripts to be functional. My one take away is how python core is not very useful, and to even remotely do anything one must have a list of imports.
      - scoopdewoop 6y ago
        'os' is in the standard library, not a concern when discussing virtual-env and third party libraries.
        dylan604 6y ago
        then why must it be imported as a 3rd party? this is my question on python in general. i get needing to explicitly import things that are not part of the core/standard, but why are core/standard required to be imported? why can't the methods they provide not just immediately available. *clearly, i've never taken a python CS style class, but it this importing core functions is just strange to me
blibble 6y ago
python is indeed very painful for typical short shell script tasks for example: anything with pipes however once you start having to run multiple background jobs with custom wait logic it probably becomes worth it
- tacitusarc 6y ago
  Python is far easier for anything involving http and data parsing/manipulation. It's far worse at running seuqntial shell programs.
- andrewshadura 6y ago
  from sh import wc, ls print(wc(ls("-1"), "-l"))
  - citrin_ru 6y ago
    Try to do in Python something like: fgrep -f keywords huge.log | cut -f 2- | sort | uniq -c | sort -rn | head Where size of data >> RAM (but less than free space in /tmp). It of course possible, but I suspect would require more effort and would work slower than a shell line above.
    - Too 6y ago
      If you want to compare memory efficiency. The sort+uniq -c would likely be more optimal in a high level language as you would there keep only the count+value in a defaultdict or alike instead of sorting the entire dataset, which requires keeping it all in mem. As for brevity, yeah, this particular example would require 5 lines instead of 1.
      - citrin_ru 6y ago
        For small datasets something like defaultdict may be more efficient, but if it would not fit in RAM you have a problem.
chasil 6y ago
There are two specific cases where you do not have BASH. 1. Ubuntu/Debian /bin/sh is the Debian variant of the Almquist shell, known as dash. The only non-POSIX element it includes is the "local" keyword (afaik). 2. Busybox, with the Almquist shell. Busybox also has a "bash" which offers a few cosmetic improvements to ash (an alias of [[ to [ is one that I can see in the source). I write POSIX shell scripts on Windows with busybox quite often. The only other shell that omits everything outside of the POSIX specification is mrsh. I don't think mrsh is widely-deployed in any major distribution. (You also aren't going to have bash in AIX/HP-UX and maybe a Solaris base load, but as the parent article says, we aren't talking about dinosaur herders.)
- saagarjha 6y ago
  tcsh was the default shell in Mac OS X for a while…
- alexhutcheson 6y ago
  For #1, you can just put #!/bin/bash at the top of the file to use Bash. Bash is still available, it’s just not the default for scripts that specify #!/bin/sh. #2 is still currently tricky, but Rob Landley (former Busybox maintainer) is working on a full bug-for-bug compatible Bash clone called toysh which will be included in an upcoming release of Toybox[1]. Once that’s released, I’m looking forward to (hopefully) never writing a script for BusyBox ash again. [1] http://landley.net/toybox/about.html http://landley.net/toybox/about.html
maddyboo 6y ago
Bash is ugly and rife with pitfalls, but it can be incredibly productive for certain classes of problems if you know what you’re doing. Trying to use Python or Go to whack some external commands together feels very cumbersome to me. For pure programming, “real” languages are absolulely preferable in almost every way, but all of them fail when it comes to running external programs and redirecting their output in a way that doesn’t make me pull my hair out. As a heavy user of both “real” programming languages and shell scripting languages, I’m left craving something that brings the best of both worlds. There are a number of newer shell projects in this vein that are very exciting, like Oil [0], Elvish [1], and Xonsh [2]. I was also hopeful that Neugram [3] would go somewhere, but the project seems to have died out. While many people cite Bash’s lack of portability as a reason not to use it, I find Bash to be very portable for my use cases and avoid using these newer shells for their lack of portability. Maybe one day we can have nice things. [0]: https://github.com/oilshell/oil https://github.com/oilshell/oil [1]: https://github.com/elves/elvish https://github.com/elves/elvish [2]: https://github.com/xonsh/xonsh https://github.com/xonsh/xonsh [3]: https://github.com/neugram/ng https://github.com/neugram/ng
- throwaway823882 6y ago
  Sad that none of those projects took the easy route. If you want to replace Bash, do it the way Bash replaced Bourne: Make it backwards compatible. Bash acts like 'historical versions of sh' if it was invoked as sh. So, a Bash replacement could act like Bash if it was invoked like bash. Then implement all the extra crazy shit if you invoke it as slash or something. Assuming it was a small, fast, compiled binary, this would take off in all the distros pretty much immediately. And if you really want it to succeed, implement a spec first, and implementations second. Add a slash implementation to Busybox, and then it's on every embedded Linux system in the world.
  - maddyboo 6y ago
    It would certainly be nice, but I wouldn’t call that the easy route. Creating a new shell that’s fully backwards compatible with the monstrosity that is Bash sounds like a massive undertaking. POSIX or the Bourne shell is probably an easier target, and there are numerous alternative shells that are POSIX-compatible and add nice new features or optional alternate syntaxes. But I agree, to properly dethrone Bash would probably require backwards compatibility with it.

lordgroff 6y ago

This used to be my take, but I'm not sure any more. Now, I don't think Python is ideal, and I am looking for something that maybe fits better than it in that gray zone of programming language vs "quick" script, but I can't see your point about debugging and bugs: bash is notoriously annoying to write bug free, and I can't see what's the issue with the Python debugger. Also if we're in that gray zone of bash vs Python, you can pretty much stick to the stdlib.

dylan604 6y ago
python3 or python2.7???
- brobinson 6y ago
  I would go with the one that hasn't been out of support since Jan 1, 2020. https://www.python.org/doc/sunset-python-2/ https://www.python.org/doc/sunset-python-2/
  - dylan604 6y ago
    yet everything comes standard with stock install of 2.7. ???
    - chungy 6y ago
      No responsible distro still ships 2.7
      - dylan604 6y ago
        hahahaha. I like your definition or responsible as AWS Linux 2 comes with 2.7 and macOS comes with 2.7 glad to know i'm such a hellion running such irresponsible OSes! down with the man!! I do what I want! Or more likely, the world isn't as clear cut as you might think
        chungy 6y ago
        > I['d] like your definition [f]or responsible A distribution that removes Python 2.7 once upstream security support has been cut. Since Python is frequently used on network-facing services, it is unconscionable to allow its use past January 2020. Heck, one might even have setuid programs in Python and you don't really want to risk an unfixed local privilege escalation either. > AWS Linux 2 A butchered version of CentOS > and macOS Not a distribution, and Apple doesn't exactly have a good track record with keeping things up to date or secure. > Or more likely, the world isn't as clear cut as you might think Given that the vast majority of the Python ecosystem is no longer compatible with 2.7, there isn't even a technical reason for kicking around 2.7 anymore. Python 2.7 is trivial to build and run if you really, really want or need it, but since all the major upstream distributions have removed it, it sends a clear signal that you must take the risks into serious consideration before doing it.
scoopdewoop 6y ago
I'm hopeful that Deno can be just that runtime. Since its a single binary, its trivial to install and script for. TBH, I don't have any technical reasons for JS/TS over python besides that I have coworkers that have never had to learn python and would prefer not to. /** * cat.ts */ for (let i = 0; i < Deno.args.length; i++) { const filename = Deno.args[i]; const file = await Deno.open(filename); await Deno.copy(file, Deno.stdout); file.close(); }
fomine3 6y ago
Ruby!

gmfawcett 6y ago

In the space of shell programming, virtualenv and third-party Python libraries are irrelevant and unnecessary.

rattray 6y ago
Huh? That's only true if you don't use any libraries, right? What if you want to use something like delegator[0] to make running shell commands less toilsome? [0] https://github.com/amitt001/delegator.py https://github.com/amitt001/delegator.py
- lmm 6y ago
  Python with no libraries is still a much, much nicer programming environment than bash.
  - dylan604 6y ago
    How say you? To even ask the system a question in python, you have to "import os". Is that not a library? I'm not a python-first type of person, but I have hacked enough scripts to be functional. My one take away is how python core is not very useful, and to even remotely do anything one must have a list of imports.
    - scoopdewoop 6y ago
      'os' is in the standard library, not a concern when discussing virtual-env and third party libraries.
      - dylan604 6y ago
        then why must it be imported as a 3rd party? this is my question on python in general. i get needing to explicitly import things that are not part of the core/standard, but why are core/standard required to be imported? why can't the methods they provide not just immediately available. *clearly, i've never taken a python CS style class, but it this importing core functions is just strange to me

blibble 6y ago

python is indeed very painful for typical short shell script tasks for example: anything with pipes however once you start having to run multiple background jobs with custom wait logic it probably becomes worth it

tacitusarc 6y ago
Python is far easier for anything involving http and data parsing/manipulation. It's far worse at running seuqntial shell programs.
andrewshadura 6y ago
from sh import wc, ls print(wc(ls("-1"), "-l"))
- citrin_ru 6y ago
  Try to do in Python something like: fgrep -f keywords huge.log | cut -f 2- | sort | uniq -c | sort -rn | head Where size of data >> RAM (but less than free space in /tmp). It of course possible, but I suspect would require more effort and would work slower than a shell line above.
  - Too 6y ago
    If you want to compare memory efficiency. The sort+uniq -c would likely be more optimal in a high level language as you would there keep only the count+value in a defaultdict or alike instead of sorting the entire dataset, which requires keeping it all in mem. As for brevity, yeah, this particular example would require 5 lines instead of 1.
    - citrin_ru 6y ago
      For small datasets something like defaultdict may be more efficient, but if it would not fit in RAM you have a problem.

chasil 6y ago

There are two specific cases where you do not have BASH. 1. Ubuntu/Debian /bin/sh is the Debian variant of the Almquist shell, known as dash. The only non-POSIX element it includes is the "local" keyword (afaik). 2. Busybox, with the Almquist shell. Busybox also has a "bash" which offers a few cosmetic improvements to ash (an alias of [[ to [ is one that I can see in the source). I write POSIX shell scripts on Windows with busybox quite often. The only other shell that omits everything outside of the POSIX specification is mrsh. I don't think mrsh is widely-deployed in any major distribution. (You also aren't going to have bash in AIX/HP-UX and maybe a Solaris base load, but as the parent article says, we aren't talking about dinosaur herders.)

saagarjha 6y ago
tcsh was the default shell in Mac OS X for a while…
alexhutcheson 6y ago
For #1, you can just put #!/bin/bash at the top of the file to use Bash. Bash is still available, it’s just not the default for scripts that specify #!/bin/sh. #2 is still currently tricky, but Rob Landley (former Busybox maintainer) is working on a full bug-for-bug compatible Bash clone called toysh which will be included in an upcoming release of Toybox[1]. Once that’s released, I’m looking forward to (hopefully) never writing a script for BusyBox ash again. [1] http://landley.net/toybox/about.html http://landley.net/toybox/about.html

maddyboo 6y ago

Bash is ugly and rife with pitfalls, but it can be incredibly productive for certain classes of problems if you know what you’re doing. Trying to use Python or Go to whack some external commands together feels very cumbersome to me. For pure programming, “real” languages are absolulely preferable in almost every way, but all of them fail when it comes to running external programs and redirecting their output in a way that doesn’t make me pull my hair out. As a heavy user of both “real” programming languages and shell scripting languages, I’m left craving something that brings the best of both worlds. There are a number of newer shell projects in this vein that are very exciting, like Oil [0], Elvish [1], and Xonsh [2]. I was also hopeful that Neugram [3] would go somewhere, but the project seems to have died out. While many people cite Bash’s lack of portability as a reason not to use it, I find Bash to be very portable for my use cases and avoid using these newer shells for their lack of portability. Maybe one day we can have nice things. [0]: https://github.com/oilshell/oil https://github.com/oilshell/oil [1]: https://github.com/elves/elvish https://github.com/elves/elvish [2]: https://github.com/xonsh/xonsh https://github.com/xonsh/xonsh [3]: https://github.com/neugram/ng https://github.com/neugram/ng

throwaway823882 6y ago
Sad that none of those projects took the easy route. If you want to replace Bash, do it the way Bash replaced Bourne: Make it backwards compatible. Bash acts like 'historical versions of sh' if it was invoked as sh. So, a Bash replacement could act like Bash if it was invoked like bash. Then implement all the extra crazy shit if you invoke it as slash or something. Assuming it was a small, fast, compiled binary, this would take off in all the distros pretty much immediately. And if you really want it to succeed, implement a spec first, and implementations second. Add a slash implementation to Busybox, and then it's on every embedded Linux system in the world.
- maddyboo 6y ago
  It would certainly be nice, but I wouldn’t call that the easy route. Creating a new shell that’s fully backwards compatible with the monstrosity that is Bash sounds like a massive undertaking. POSIX or the Bourne shell is probably an easier target, and there are numerous alternative shells that are POSIX-compatible and add nice new features or optional alternate syntaxes. But I agree, to properly dethrone Bash would probably require backwards compatibility with it.