5 ms·

When I see very long bash scripts I get angry. There is a reason why perl & later python were invented and they are both in 98% of cases available on Unix compa

When I see very long bash scripts I get angry. There is a reason why perl & later python were invented and they are both in 98% of cases available on Unix compatible systems. Not that I'm not ok with short bash scripts or for systems were you only have bash... but if your Unix compatible system only has bash there is something else wrong with your system in my point of view.

gjulianm 9y ago
Python and perl are not always the best option. For example, if I am executing a lot of system commands (sometimes through ssh), checking outputs with grep/awk, reading files, calling make, why use a language that is going to be far more verbose, difficult to modify, and show problems because it's not executing the shell in the correct profile or things like that? For interacting with the system, there's nothing better than shell scripts. You mainly only have to take care with the quotes to avoid word separation and globbing, and use the *nix commands to avoid recreating the wheel.
- scaryclam 9y ago
  I'll second this. My Python is far better than my bash but there's no point writing more complicated Python if you're working with system commands. That said, quite often I'll break non-system stuff out into python scripts and wrap them in bash if I need to do a bit of both.
arghwhat 9y ago
I'd much rather have 200 lines of bash than the 300-400 lines of python to replace it. I have recently rewritten a lot of Python and Perl (over 10kLOC) to a small collectiom of 10-ish bash scripts that weigh in at about 700LOC total. There are some tasks where Bash is the only appropriate tool. Much simpler code, much easier to read, and a lot faster too.
- omribahumi 9y ago
  How is bash faster than Python?
  - arghwhat 9y ago
    When what you need is a pipe, Python won't do anything positive to performance. Plus, CPython is horrendously slow in general. Other than that, the simpler design also allowed greater parallelism.
  - barrkel 9y ago
    Bash lets you glue best-of-breed tools together, as long as they all talk on stdin / stdout / files. The tools themselves can be written in the best language for their task.
- growse 9y ago
  If you're measuring code quality / maintainability by how many lines of code there are, you're doing it wrong. I'd much rather see 400 lines of well-structured python than 200 lines of tangled bash. For the rewrite you did, how much of the benefit was because it was rewritten, rather than being rewritten _in bash_? I'd also go as far as saying that if you're using bash because it's faster than python, you're also using the wrong tool for the job.
  - arghwhat 9y ago
    The quality of the bash far outshined the quality of the python and perl, because Python and Perl was being used to do a bash scripts job. When what you need is to call a bunch of binaries and redirect output around, then Python and Perl just don't cut it. The code is much, much easier to read now. You can see a script on a single screen height, which works wonders for overview. The 3x performance gain spawned from enlightenment this overview granted us. But don't get me wrong, in the very same rewrite, I rewrote 100 lines of bash + 300 lines of Python into a single 200 lines of Python, which when combined with PyPy, was 50 times faster than the original setup. Python doesn't do bash's job very well, and bash doesn't do Python's job very well either.
    - growse 9y ago
      I agree that for the simple job of "start process, maybe echo some stuff", then bash is superior to python. However, my difficulty always arises the moment you want to start writing 'if', or handle some sort of error condition. Once you go down that road, you're into magic switches at the top of your script (which TFA demonstrates aren't well understood), and subtle control issues where bash's flow control doesn't quite do what you think it should, and (for me) this is where bash stops being superior.
      - arghwhat 9y ago
        I don't think flow control is a problem, as long as it's just to complete a simple purpose. Checking if a file is present, running some code for every file or every line, etc. But bash is definitely not something where complicated logic belongs.
arjie 9y ago
People always recommend Perl over a shell script but I've never seen a situation where it makes sense. Having written very little Perl, it seems like its only strength is that it's present everywhere and has a large library. Language-wise, Python is far superior and has a large ecosystem and these days it has ubiquity too. I'd be interested in the argument for Perl over shell scripts that does not apply to just writing the program in a modern language.
- therealmarv 9y ago
  Have you ever done string manipulation (also RegEx) over many files in Perl? I think it's in this regard superior than anything else, even python when looking at syntax.
  - Outpox 9y ago
    For someone who hasn't done any Perl yet and only little Python could you explain why?
    - mfontani 9y ago
      For the specific thing the OP mentioned, this: $ perl -pi -e 's/MY_NAME/Marco/g' ./*.txt ... would go through all "*.txt" files in the directory, and globally replace the token "MY_NAME" with "Marco". Great for a {poor man's,simple and easy to understand} basic template system to be used inside a bash script ;)
      - JetSpiegel 9y ago
        Overkill. Just use: $ sed -i 's,MY_NAME,Marco,g' *.txt
        therealmarv 9y ago
        nope. I've dealt recently with some big DB files (I think GB size) and regex doing the same. sed is a lot slower than perl for this kind of task.
yjftsjthsd-h 9y ago
This is the first time I have seen anoybody hold up perl as the superior option. I don't doubt that it will work, but I need the next admin to be able to read my code. Also, every system may have some Python installed, bud by the time I've dealt with versions (2.6 is the CentOS 6 default, 3.6 is the current version) and dependencies (virtualenv? system wide pip? PYTHONPATH? RPM Python modules?) I'd better get a significant improvement.

gjulianm 9y ago

Python and perl are not always the best option. For example, if I am executing a lot of system commands (sometimes through ssh), checking outputs with grep/awk, reading files, calling make, why use a language that is going to be far more verbose, difficult to modify, and show problems because it's not executing the shell in the correct profile or things like that? For interacting with the system, there's nothing better than shell scripts. You mainly only have to take care with the quotes to avoid word separation and globbing, and use the *nix commands to avoid recreating the wheel.

scaryclam 9y ago
I'll second this. My Python is far better than my bash but there's no point writing more complicated Python if you're working with system commands. That said, quite often I'll break non-system stuff out into python scripts and wrap them in bash if I need to do a bit of both.

arghwhat 9y ago

I'd much rather have 200 lines of bash than the 300-400 lines of python to replace it. I have recently rewritten a lot of Python and Perl (over 10kLOC) to a small collectiom of 10-ish bash scripts that weigh in at about 700LOC total. There are some tasks where Bash is the only appropriate tool. Much simpler code, much easier to read, and a lot faster too.

omribahumi 9y ago
How is bash faster than Python?
- arghwhat 9y ago
  When what you need is a pipe, Python won't do anything positive to performance. Plus, CPython is horrendously slow in general. Other than that, the simpler design also allowed greater parallelism.
- barrkel 9y ago
  Bash lets you glue best-of-breed tools together, as long as they all talk on stdin / stdout / files. The tools themselves can be written in the best language for their task.
growse 9y ago
If you're measuring code quality / maintainability by how many lines of code there are, you're doing it wrong. I'd much rather see 400 lines of well-structured python than 200 lines of tangled bash. For the rewrite you did, how much of the benefit was because it was rewritten, rather than being rewritten _in bash_? I'd also go as far as saying that if you're using bash because it's faster than python, you're also using the wrong tool for the job.
- arghwhat 9y ago
  The quality of the bash far outshined the quality of the python and perl, because Python and Perl was being used to do a bash scripts job. When what you need is to call a bunch of binaries and redirect output around, then Python and Perl just don't cut it. The code is much, much easier to read now. You can see a script on a single screen height, which works wonders for overview. The 3x performance gain spawned from enlightenment this overview granted us. But don't get me wrong, in the very same rewrite, I rewrote 100 lines of bash + 300 lines of Python into a single 200 lines of Python, which when combined with PyPy, was 50 times faster than the original setup. Python doesn't do bash's job very well, and bash doesn't do Python's job very well either.
  - growse 9y ago
    I agree that for the simple job of "start process, maybe echo some stuff", then bash is superior to python. However, my difficulty always arises the moment you want to start writing 'if', or handle some sort of error condition. Once you go down that road, you're into magic switches at the top of your script (which TFA demonstrates aren't well understood), and subtle control issues where bash's flow control doesn't quite do what you think it should, and (for me) this is where bash stops being superior.
    - arghwhat 9y ago
      I don't think flow control is a problem, as long as it's just to complete a simple purpose. Checking if a file is present, running some code for every file or every line, etc. But bash is definitely not something where complicated logic belongs.

arjie 9y ago

People always recommend Perl over a shell script but I've never seen a situation where it makes sense. Having written very little Perl, it seems like its only strength is that it's present everywhere and has a large library. Language-wise, Python is far superior and has a large ecosystem and these days it has ubiquity too. I'd be interested in the argument for Perl over shell scripts that does not apply to just writing the program in a modern language.

therealmarv 9y ago
Have you ever done string manipulation (also RegEx) over many files in Perl? I think it's in this regard superior than anything else, even python when looking at syntax.
- Outpox 9y ago
  For someone who hasn't done any Perl yet and only little Python could you explain why?
  - mfontani 9y ago
    For the specific thing the OP mentioned, this: $ perl -pi -e 's/MY_NAME/Marco/g' ./*.txt ... would go through all "*.txt" files in the directory, and globally replace the token "MY_NAME" with "Marco". Great for a {poor man's,simple and easy to understand} basic template system to be used inside a bash script ;)
    - JetSpiegel 9y ago
      Overkill. Just use: $ sed -i 's,MY_NAME,Marco,g' *.txt
      - therealmarv 9y ago
        nope. I've dealt recently with some big DB files (I think GB size) and regex doing the same. sed is a lot slower than perl for this kind of task.

yjftsjthsd-h 9y ago

This is the first time I have seen anoybody hold up perl as the superior option. I don't doubt that it will work, but I need the next admin to be able to read my code. Also, every system may have some Python installed, bud by the time I've dealt with versions (2.6 is the CentOS 6 default, 3.6 is the current version) and dependencies (virtualenv? system wide pip? PYTHONPATH? RPM Python modules?) I'd better get a significant improvement.