5 ms·

How about you don't? Bash as scripting language is rather mediocre. Anything that is not simple in bash gets hard to read and debug and probably is wrong on so

by bhaak 9y ago

How about you don't? Bash as scripting language is rather mediocre. Anything that is not simple in bash gets hard to read and debug and probably is wrong on some subtle levels. I have a rule of thumb that any shell script that grows beyond a screenful of lines gets redone in a proper scripting language.

jalanb 9y ago
How about OP does anyway? Bash as a scripting language is fucking great! Since when is simplicity an argument against writing programs? Whether scripts or frameworks? "Hard to read" is not neccessarily an inherent trait[1] of the language, and more likely wrong on some PEBKAC level. I have a customised environment at near 10k lines of bash in 5 projects, all of it in the correct tool for the job, aka a proper scripting language, so I can suggest another use for your thumb :-) 1: https://www.reddit.com/r/commandline/comments/2kq8oa/the_most_productive_function_i_have_written/clo0gh2/ https://www.reddit.com/r/commandline/comments/2kq8oa/the_mos...
barrkel 9y ago
OTOH, if you can structure your problem as a composition of pipelines, it can be quite a bit faster in bash than in a "proper" language. You get to choose optimized tools, and they run concurrently. Writing efficient bash code forces you to think about your problem differently. It's a very similar process to thinking functionally; e.g. you don't want to deal with lines of a file one at a time in a loop, you want to do filters and maps in languages like grep, sed and awk to deal with data in a streaming fashion with a minimum of forked processes.
- oblio 9y ago
  Then you have the age old problem: what if one of the tasks in your pipeline(s) fails?
  - yorwba 9y ago
    set -o pipefail -o errexit Add more options to taste.
    - Benjamin_Dobell 9y ago
      https://stackoverflow.com/questions/29532904/bash-subshell-errexit-semantics/29540745#29540745 https://stackoverflow.com/questions/29532904/bash-subshell-e... Not pipefail as such, but Bash's error handling semantics (or lack there of) is pretty lame.
- fiatjaf 9y ago
  Is there a "proper scripting language" somewhere that supports the same first-class access to Unix programs and the same piping | syntax that shells in general do? I would love something like that.
  - soveran 9y ago
    In most platforms you can install the rc shell from plan9. It's what I use exclusively for my shell scripts. It is only when I want to share some script with other people that I consider using /bin/sh, and even then I've gone for rc nevertheless. Here you can read about it: http://doc.cat-v.org/plan_9/4th_edition/papers/rc http://doc.cat-v.org/plan_9/4th_edition/papers/rc
    - chriswarbo 9y ago
      Thanks for the suggestion. I've also been after a saner shell, and have been disappointed in one way or another with the approach of either using another language idiomatically (Python, Scheme, Haskell, Scala, etc.), since running commands, piping, etc. are all rather awkward; or using such languages with embedded shell-like libraries, which seem to have awkward edge-cases. As a "real" shell, it looks like rc maintains the command, file and piping niceties of bash, whilst avoiding the edge-cases of shell-like embeddings.
  - fiatjaf 9y ago
    Just found out that xon.sh http://xon.sh/tutorial http://xon.sh/tutorial . html does this.
  - reacweb 9y ago
    Perl of course, even if the adjective "proper" seems weird for Perl ;-) As a rule of thumb, if a script starts to need arrays or to handle spaces in filename, I migrate it to Perl. Error handling is also easier and more natural in Perl.
  - jerf 9y ago
    In Perl, IPC::Run [1], while a bit on the quirky side sometimes, can do that sort of thing. use IPC::Run qw(run); IPC::Run::run(["echo", "abc"], "|", ["tr", "a-z", "A-Z"], "|", ["cat"], \$out); print "out: $out"; will yield out: ABC Useless use of cat here just to make a point, of course. It's easy to use Perl strings as input or output, or files, and there's also ways to interact with streams as they go along. Generally speaking the slightly more verbose interaction with shell is made up for by the savings when you can use Perl directly to do something rather than spawn a shell process for something simple. (Shell composition can be powerful, yes, but on the other hand spawning a full process and setting up a pipe for things like "tr" or "wc" is often just silly.) I also personally believe there's a win in the syntax; you may say "What? 'tr a-z A-Z' is way more convenient than '["tr", "a-z", "A-Z"]' but I say one of the biggest and most pervasive errors in shell is to incorrectly set up the arguments by having something interpolated in incorrectly. Having an unambiguous syntax for separating arguments has often made it much safer for me to write the code that has to use the shell. Used correctly it can even be made to work in the face of user-supplied input, something that should generally not be combined with any form of implicit argument separation. (Although bear in mind that "used correctly" encompasses more than just "separating the arguments correctly.") I am also NOT claiming exclusivity; this is just the thing I know off the top of my head. I'm sure the other scripting languages have nice libraries, too. Though watch out for them being too nice. There's a bit of an impedance mismatch between shell and scripting language. Anything that smooths it over too well is probably either giving up critical features or introducing subtle bugs, which can become security bugs in the face of arbitrary user input. [1]: http://search.cpan.org/~toddr/IPC-Run-0.96/lib/IPC/Run.pm http://search.cpan.org/~toddr/IPC-Run-0.96/lib/IPC/Run.pm
pjc50 9y ago
Indeed. While you can get the rules regarding file name escaping right, the language does nothing to help you get it right every time, so unless you're careful someone will leave a file with a space (or, God help you, a newline) in the name in exactly the wrong place and blow it up.
gpderetta 9y ago
I would say that a screenful is already on the large side. Then again, there is a lot you can do in a screenful in bash.
periegetes 9y ago
Bash as a scripting language is actually pretty amazing. It gives you everything you need to perform some quick-and-dirty tasks with minimal overhead. If you need only work on sequential data, files and processes, it's a perfect match. It's not a full-fledged programming language by any stretch of the imagination (lacking structures more complex than associative arrays), but it's damn good for scripts of all sorts. As an example, I've reimplemented a subset of Ansible (a command able to send "modules" on multiple machines via SSH and capturing+caching their output for subsequent queries) in ~150 lines of Bash. Considering that the size of Ansible, written in the more proper Python, is ~15000 LOC, I'd say Python is the much lesser scripting language. Edit: to answer the OP's question, the documentation I've found most helpful to learn Bash is the one present on the Linux Documentation Project, with the page for arrays deserving special mention : http://tldp.org/LDP/abs/html/arrays.html http://tldp.org/LDP/abs/html/arrays.html. I spent a lot of time reading the manual before stumbling upon that documentation, and none of it really clicked until I had a few examples before my eyes.
- dkns 9y ago
  Ansible has to take into account numerous edge cases, operating systems, backwards compatibility, etc. etc. Of course it's much, much bigger than your 150 lines bash script.
- ribs 9y ago
  Typically a program like Ansible will have the majority of its use cases implemented in a minority of its code, while the rest of the code will be there to support special cases, edge cases, rare use cases, etc. So that contributes to the disparity in size.
amelius 9y ago
Totally agree. Also, even if you manage to become better in Bash, you are bound to lose your skills at some point when you have been programming in other languages for a while. I always have to look up how to do even basic things in Bash. I just don't use it often enough for these things to "stick".
- milkytron 9y ago
  It's okay to not have everything memorized. As long as you know what to look for, you're only a google search away.
radiospiel 9y ago
If you have really limited resources, then bash or one of the other shells are the only tools at hand. Embedded devices might not give you enough disk space to get perl, ruby or python.
Diti 9y ago
There is this one case for which shell scripting is particularly interesting: dependencies simplification. https://github.com/EtiennePerot/parcimonie.sh/blob/48044f9134f91c6e86b132755603973d7623455f/README.md#user-content-why-a-reimplementation https://github.com/EtiennePerot/parcimonie.sh/blob/48044f913...

by bhaak 9y ago

jalanb 9y ago

How about OP does anyway? Bash as a scripting language is fucking great! Since when is simplicity an argument against writing programs? Whether scripts or frameworks? "Hard to read" is not neccessarily an inherent trait[1] of the language, and more likely wrong on some PEBKAC level. I have a customised environment at near 10k lines of bash in 5 projects, all of it in the correct tool for the job, aka a proper scripting language, so I can suggest another use for your thumb :-) 1: https://www.reddit.com/r/commandline/comments/2kq8oa/the_most_productive_function_i_have_written/clo0gh2/ https://www.reddit.com/r/commandline/comments/2kq8oa/the_mos...

barrkel 9y ago

OTOH, if you can structure your problem as a composition of pipelines, it can be quite a bit faster in bash than in a "proper" language. You get to choose optimized tools, and they run concurrently. Writing efficient bash code forces you to think about your problem differently. It's a very similar process to thinking functionally; e.g. you don't want to deal with lines of a file one at a time in a loop, you want to do filters and maps in languages like grep, sed and awk to deal with data in a streaming fashion with a minimum of forked processes.

oblio 9y ago
Then you have the age old problem: what if one of the tasks in your pipeline(s) fails?
- yorwba 9y ago
  set -o pipefail -o errexit Add more options to taste.
  - Benjamin_Dobell 9y ago
    https://stackoverflow.com/questions/29532904/bash-subshell-errexit-semantics/29540745#29540745 https://stackoverflow.com/questions/29532904/bash-subshell-e... Not pipefail as such, but Bash's error handling semantics (or lack there of) is pretty lame.
fiatjaf 9y ago
Is there a "proper scripting language" somewhere that supports the same first-class access to Unix programs and the same piping | syntax that shells in general do? I would love something like that.
- soveran 9y ago
  In most platforms you can install the rc shell from plan9. It's what I use exclusively for my shell scripts. It is only when I want to share some script with other people that I consider using /bin/sh, and even then I've gone for rc nevertheless. Here you can read about it: http://doc.cat-v.org/plan_9/4th_edition/papers/rc http://doc.cat-v.org/plan_9/4th_edition/papers/rc
  - chriswarbo 9y ago
    Thanks for the suggestion. I've also been after a saner shell, and have been disappointed in one way or another with the approach of either using another language idiomatically (Python, Scheme, Haskell, Scala, etc.), since running commands, piping, etc. are all rather awkward; or using such languages with embedded shell-like libraries, which seem to have awkward edge-cases. As a "real" shell, it looks like rc maintains the command, file and piping niceties of bash, whilst avoiding the edge-cases of shell-like embeddings.
- fiatjaf 9y ago
  Just found out that xon.sh http://xon.sh/tutorial http://xon.sh/tutorial . html does this.
- reacweb 9y ago
  Perl of course, even if the adjective "proper" seems weird for Perl ;-) As a rule of thumb, if a script starts to need arrays or to handle spaces in filename, I migrate it to Perl. Error handling is also easier and more natural in Perl.
- jerf 9y ago
  In Perl, IPC::Run [1], while a bit on the quirky side sometimes, can do that sort of thing. use IPC::Run qw(run); IPC::Run::run(["echo", "abc"], "|", ["tr", "a-z", "A-Z"], "|", ["cat"], \$out); print "out: $out"; will yield out: ABC Useless use of cat here just to make a point, of course. It's easy to use Perl strings as input or output, or files, and there's also ways to interact with streams as they go along. Generally speaking the slightly more verbose interaction with shell is made up for by the savings when you can use Perl directly to do something rather than spawn a shell process for something simple. (Shell composition can be powerful, yes, but on the other hand spawning a full process and setting up a pipe for things like "tr" or "wc" is often just silly.) I also personally believe there's a win in the syntax; you may say "What? 'tr a-z A-Z' is way more convenient than '["tr", "a-z", "A-Z"]' but I say one of the biggest and most pervasive errors in shell is to incorrectly set up the arguments by having something interpolated in incorrectly. Having an unambiguous syntax for separating arguments has often made it much safer for me to write the code that has to use the shell. Used correctly it can even be made to work in the face of user-supplied input, something that should generally not be combined with any form of implicit argument separation. (Although bear in mind that "used correctly" encompasses more than just "separating the arguments correctly.") I am also NOT claiming exclusivity; this is just the thing I know off the top of my head. I'm sure the other scripting languages have nice libraries, too. Though watch out for them being too nice. There's a bit of an impedance mismatch between shell and scripting language. Anything that smooths it over too well is probably either giving up critical features or introducing subtle bugs, which can become security bugs in the face of arbitrary user input. [1]: http://search.cpan.org/~toddr/IPC-Run-0.96/lib/IPC/Run.pm http://search.cpan.org/~toddr/IPC-Run-0.96/lib/IPC/Run.pm

pjc50 9y ago

Indeed. While you can get the rules regarding file name escaping right, the language does nothing to help you get it right every time, so unless you're careful someone will leave a file with a space (or, God help you, a newline) in the name in exactly the wrong place and blow it up.

gpderetta 9y ago

I would say that a screenful is already on the large side. Then again, there is a lot you can do in a screenful in bash.

periegetes 9y ago

Bash as a scripting language is actually pretty amazing. It gives you everything you need to perform some quick-and-dirty tasks with minimal overhead. If you need only work on sequential data, files and processes, it's a perfect match. It's not a full-fledged programming language by any stretch of the imagination (lacking structures more complex than associative arrays), but it's damn good for scripts of all sorts. As an example, I've reimplemented a subset of Ansible (a command able to send "modules" on multiple machines via SSH and capturing+caching their output for subsequent queries) in ~150 lines of Bash. Considering that the size of Ansible, written in the more proper Python, is ~15000 LOC, I'd say Python is the much lesser scripting language. Edit: to answer the OP's question, the documentation I've found most helpful to learn Bash is the one present on the Linux Documentation Project, with the page for arrays deserving special mention : http://tldp.org/LDP/abs/html/arrays.html http://tldp.org/LDP/abs/html/arrays.html. I spent a lot of time reading the manual before stumbling upon that documentation, and none of it really clicked until I had a few examples before my eyes.

dkns 9y ago
Ansible has to take into account numerous edge cases, operating systems, backwards compatibility, etc. etc. Of course it's much, much bigger than your 150 lines bash script.
ribs 9y ago
Typically a program like Ansible will have the majority of its use cases implemented in a minority of its code, while the rest of the code will be there to support special cases, edge cases, rare use cases, etc. So that contributes to the disparity in size.

amelius 9y ago

Totally agree. Also, even if you manage to become better in Bash, you are bound to lose your skills at some point when you have been programming in other languages for a while. I always have to look up how to do even basic things in Bash. I just don't use it often enough for these things to "stick".

milkytron 9y ago
It's okay to not have everything memorized. As long as you know what to look for, you're only a google search away.

radiospiel 9y ago

If you have really limited resources, then bash or one of the other shells are the only tools at hand. Embedded devices might not give you enough disk space to get perl, ruby or python.

Diti 9y ago

There is this one case for which shell scripting is particularly interesting: dependencies simplification. https://github.com/EtiennePerot/parcimonie.sh/blob/48044f9134f91c6e86b132755603973d7623455f/README.md#user-content-why-a-reimplementation https://github.com/EtiennePerot/parcimonie.sh/blob/48044f913...