4 ms·

I'd much rather have 200 lines of bash than the 300-400 lines of python to replace it. I have recently rewritten a lot of Python and Perl (over 10kLOC) to a sm

I'd much rather have 200 lines of bash than the 300-400 lines of python to replace it. I have recently rewritten a lot of Python and Perl (over 10kLOC) to a small collectiom of 10-ish bash scripts that weigh in at about 700LOC total. There are some tasks where Bash is the only appropriate tool. Much simpler code, much easier to read, and a lot faster too.

omribahumi 10y ago
How is bash faster than Python?
- arghwhat 10y ago
  When what you need is a pipe, Python won't do anything positive to performance. Plus, CPython is horrendously slow in general. Other than that, the simpler design also allowed greater parallelism.
- barrkel 10y ago
  Bash lets you glue best-of-breed tools together, as long as they all talk on stdin / stdout / files. The tools themselves can be written in the best language for their task.
growse 10y ago
If you're measuring code quality / maintainability by how many lines of code there are, you're doing it wrong. I'd much rather see 400 lines of well-structured python than 200 lines of tangled bash. For the rewrite you did, how much of the benefit was because it was rewritten, rather than being rewritten _in bash_? I'd also go as far as saying that if you're using bash because it's faster than python, you're also using the wrong tool for the job.
- arghwhat 10y ago
  The quality of the bash far outshined the quality of the python and perl, because Python and Perl was being used to do a bash scripts job. When what you need is to call a bunch of binaries and redirect output around, then Python and Perl just don't cut it. The code is much, much easier to read now. You can see a script on a single screen height, which works wonders for overview. The 3x performance gain spawned from enlightenment this overview granted us. But don't get me wrong, in the very same rewrite, I rewrote 100 lines of bash + 300 lines of Python into a single 200 lines of Python, which when combined with PyPy, was 50 times faster than the original setup. Python doesn't do bash's job very well, and bash doesn't do Python's job very well either.
  - growse 10y ago
    I agree that for the simple job of "start process, maybe echo some stuff", then bash is superior to python. However, my difficulty always arises the moment you want to start writing 'if', or handle some sort of error condition. Once you go down that road, you're into magic switches at the top of your script (which TFA demonstrates aren't well understood), and subtle control issues where bash's flow control doesn't quite do what you think it should, and (for me) this is where bash stops being superior.
    - arghwhat 10y ago
      I don't think flow control is a problem, as long as it's just to complete a simple purpose. Checking if a file is present, running some code for every file or every line, etc. But bash is definitely not something where complicated logic belongs.
  - edejong 10y ago
    Although it's been a long time since I last did perl programming, it has always been my understanding that perl is particularly well designed for piping and mangling input and output from binaries. It's been the successor of awk, sed, grep and cut, all contained in one, relatively consistent and well documented language. Are you saying Larry Wall failed in this design attempt? Could it be that Perl wasn't used to its fullest potential?
    - mfontani 10y ago
      Perl is great at "mangling input", absolutely. But doing stuff like executing commands with standard output or standard error redirection, with pipes, etc. is something it's not strong at. That's what a shell script is strong at. To clarify, I'd: use bash to create an execution pipeline, which itself may contain a Perl script which munges some output. Bash in this case is more of a "pipeline and execution orchestrator", and Perl is used to "properly munge the output" to make stuff happen.
      - jerf 10y ago

omribahumi 10y ago

How is bash faster than Python?

arghwhat 10y ago
When what you need is a pipe, Python won't do anything positive to performance. Plus, CPython is horrendously slow in general. Other than that, the simpler design also allowed greater parallelism.
barrkel 10y ago
Bash lets you glue best-of-breed tools together, as long as they all talk on stdin / stdout / files. The tools themselves can be written in the best language for their task.

growse 10y ago

If you're measuring code quality / maintainability by how many lines of code there are, you're doing it wrong. I'd much rather see 400 lines of well-structured python than 200 lines of tangled bash. For the rewrite you did, how much of the benefit was because it was rewritten, rather than being rewritten _in bash_? I'd also go as far as saying that if you're using bash because it's faster than python, you're also using the wrong tool for the job.

arghwhat 10y ago
The quality of the bash far outshined the quality of the python and perl, because Python and Perl was being used to do a bash scripts job. When what you need is to call a bunch of binaries and redirect output around, then Python and Perl just don't cut it. The code is much, much easier to read now. You can see a script on a single screen height, which works wonders for overview. The 3x performance gain spawned from enlightenment this overview granted us. But don't get me wrong, in the very same rewrite, I rewrote 100 lines of bash + 300 lines of Python into a single 200 lines of Python, which when combined with PyPy, was 50 times faster than the original setup. Python doesn't do bash's job very well, and bash doesn't do Python's job very well either.
- growse 10y ago
  I agree that for the simple job of "start process, maybe echo some stuff", then bash is superior to python. However, my difficulty always arises the moment you want to start writing 'if', or handle some sort of error condition. Once you go down that road, you're into magic switches at the top of your script (which TFA demonstrates aren't well understood), and subtle control issues where bash's flow control doesn't quite do what you think it should, and (for me) this is where bash stops being superior.
  - arghwhat 10y ago
    I don't think flow control is a problem, as long as it's just to complete a simple purpose. Checking if a file is present, running some code for every file or every line, etc. But bash is definitely not something where complicated logic belongs.
- edejong 10y ago
  Although it's been a long time since I last did perl programming, it has always been my understanding that perl is particularly well designed for piping and mangling input and output from binaries. It's been the successor of awk, sed, grep and cut, all contained in one, relatively consistent and well documented language. Are you saying Larry Wall failed in this design attempt? Could it be that Perl wasn't used to its fullest potential?
  - mfontani 10y ago
    Perl is great at "mangling input", absolutely. But doing stuff like executing commands with standard output or standard error redirection, with pipes, etc. is something it's not strong at. That's what a shell script is strong at. To clarify, I'd: use bash to create an execution pipeline, which itself may contain a Perl script which munges some output. Bash in this case is more of a "pipeline and execution orchestrator", and Perl is used to "properly munge the output" to make stuff happen.
    - jerf 10y ago