3 ms·

Perl is great at "mangling input", absolutely. But doing stuff like executing commands with standard output or standard error redirection, with pipes, etc. is

Perl is great at "mangling input", absolutely. But doing stuff like executing commands with standard output or standard error redirection, with pipes, etc. is something it's not strong at. That's what a shell script is strong at. To clarify, I'd: use bash to create an execution pipeline, which itself may contain a Perl script which munges some output. Bash in this case is more of a "pipeline and execution orchestrator", and Perl is used to "properly munge the output" to make stuff happen.

jerf 9y ago
use IPC::Run qw(run); run(["cat", "/etc/passwd"], "|", ["egrep", "-v", ':/bin/bash$'], "|", ["wc", "-l"], undef, \$out); print $out; Not the best way to do that sort of thing (useless use of cat for instance), I was just trying to come up with an executable example. run returns success or failure of the pipeline as the return code, which I don't show here. I don't think you can pick out which command failed. I also find that once you're in a real programming language, you have much less need to have deep pipelines anyhow. Consider the pure-perl equivalent open PW, "<", "/etc/passwd" or die; while (<PW>) { next if $_ =~ m|:/bin/bash$|; $count++; } print "$count\n"; Of course "real perl" would suggest some form of "use strict" and the corresponding need to declare the $count var, but if we're discussing shell script contexts it is at least fair to consider ignoring that, since it would be a silly objection that the perl equivalent doesn't turn on checks that shell scripting doesn't even have. But if you're in a real programming language you don't need to shell out to wc just to get a line count, or write complicated awk scripts, etc. It's much more common for me in those cases to execute a single program and control its STDIN/STDOUT/STDERR and watch its exit code.
- arghwhat 9y ago
  I am very much against ever shelling out from an application. Either write a shell script, or a proper application. Nothing bugs me more than shell scripts written in Python, Perl, or even gasp C++ (he was later fired). However, most bash scripts I interact with will take at least 2-4 times the amount of code to replicate as a proper application. The line count example is a good pro-shell one, with 6 not particularly nice lines of code replacing a single, simple line: egrep -v ':/bin/bash$' /etc/passwd | wc -l "Complicated" AWK scripts are also much more compact than their equivalent. Say, `xyz | awk '/thing/ { getline; print $0; }'` to get the line after a pattern. I wouldn't write an AWK script much more complicated than that, though, and for me, Perl and AWK belong in the same bucket.
  - jerf 9y ago
    My objection to shell is that it obtains that line advantage by cheating. It has terribly poor error handling, it uses an incredibly sloppy serialization format that frequently breaks at the scales people attempt to apply it at, its performance characteristics are terrible, its debugging story is nonexistant, perhaps worst of all it affords a programming style that blinds people to how much compromise they are making with their data because of the aforementioned serialization format, and on it goes. Of course programming languages can't get as concise as shell; even the sloppiest programming language communities there are have the bar for quality set substantially above what shell can meet. Your example isn't even equivalent to my rather sloppy Perl, since, for instance, it will do something quite different if /etc/passwd doesn't exist. Shell only has the size advantage as long as everything goes perfectly correctly. Shell is only good for two cases: 1. You don't much care about the integrity of your input or output, and you don't much care about what happens if something goes wrong. 2. When you don't care about the aforementioned issues because you're right there on the spot and can fix things if they do go wrong, because you're using shell interactively. Having fiddled around with designing a "new shell" every so often my current conclusion is that there is no way to bridge the gap between the optimal interactive use shell and the optimal shell script with one language, and modern shell scripting, for all the features they have apparently for shell scripts, are firmly for the interactive case.
    - arghwhat 9y ago
      My example is still very well defined, and very easy to reason about. It will output '0' on stdout if no file is found, and an error message on stderr. Bash is very good for the things its good at, but you still need to know what you're doing to not fuck it up. Some people don't know how to consider error handling when they code, but that's the fault of the programmer.
      - jerf 9y ago
        I used to believe that. I don't any longer. Your second paragraph basically agrees with me that bash affords bad code that ignores errors. I think it's past time for the programming world to take that seriously. We've had 60-some years of programmers insisting they can heroically just remember all the things they need to do to write correct code. And they are observably wrong. To disagree with me on that is basically to assert that software is nearly universally of a high quality and generally robust to all reasonably forseeable error conditions. Shell goes even beyond that, though; as evidenced by articles like this you can't even get good agreement on how you should write safe shell. When even experts can't agree on what's safe, that's just something unsuitable to any serious task.

jerf 9y ago

use IPC::Run qw(run); run(["cat", "/etc/passwd"], "|", ["egrep", "-v", ':/bin/bash$'], "|", ["wc", "-l"], undef, \$out); print $out; Not the best way to do that sort of thing (useless use of cat for instance), I was just trying to come up with an executable example. run returns success or failure of the pipeline as the return code, which I don't show here. I don't think you can pick out which command failed. I also find that once you're in a real programming language, you have much less need to have deep pipelines anyhow. Consider the pure-perl equivalent open PW, "<", "/etc/passwd" or die; while (<PW>) { next if $_ =~ m|:/bin/bash$|; $count++; } print "$count\n"; Of course "real perl" would suggest some form of "use strict" and the corresponding need to declare the $count var, but if we're discussing shell script contexts it is at least fair to consider ignoring that, since it would be a silly objection that the perl equivalent doesn't turn on checks that shell scripting doesn't even have. But if you're in a real programming language you don't need to shell out to wc just to get a line count, or write complicated awk scripts, etc. It's much more common for me in those cases to execute a single program and control its STDIN/STDOUT/STDERR and watch its exit code.

arghwhat 9y ago
I am very much against ever shelling out from an application. Either write a shell script, or a proper application. Nothing bugs me more than shell scripts written in Python, Perl, or even gasp C++ (he was later fired). However, most bash scripts I interact with will take at least 2-4 times the amount of code to replicate as a proper application. The line count example is a good pro-shell one, with 6 not particularly nice lines of code replacing a single, simple line: egrep -v ':/bin/bash$' /etc/passwd | wc -l "Complicated" AWK scripts are also much more compact than their equivalent. Say, `xyz | awk '/thing/ { getline; print $0; }'` to get the line after a pattern. I wouldn't write an AWK script much more complicated than that, though, and for me, Perl and AWK belong in the same bucket.
- jerf 9y ago
  My objection to shell is that it obtains that line advantage by cheating. It has terribly poor error handling, it uses an incredibly sloppy serialization format that frequently breaks at the scales people attempt to apply it at, its performance characteristics are terrible, its debugging story is nonexistant, perhaps worst of all it affords a programming style that blinds people to how much compromise they are making with their data because of the aforementioned serialization format, and on it goes. Of course programming languages can't get as concise as shell; even the sloppiest programming language communities there are have the bar for quality set substantially above what shell can meet. Your example isn't even equivalent to my rather sloppy Perl, since, for instance, it will do something quite different if /etc/passwd doesn't exist. Shell only has the size advantage as long as everything goes perfectly correctly. Shell is only good for two cases: 1. You don't much care about the integrity of your input or output, and you don't much care about what happens if something goes wrong. 2. When you don't care about the aforementioned issues because you're right there on the spot and can fix things if they do go wrong, because you're using shell interactively. Having fiddled around with designing a "new shell" every so often my current conclusion is that there is no way to bridge the gap between the optimal interactive use shell and the optimal shell script with one language, and modern shell scripting, for all the features they have apparently for shell scripts, are firmly for the interactive case.
  - arghwhat 9y ago
    My example is still very well defined, and very easy to reason about. It will output '0' on stdout if no file is found, and an error message on stderr. Bash is very good for the things its good at, but you still need to know what you're doing to not fuck it up. Some people don't know how to consider error handling when they code, but that's the fault of the programmer.
    - jerf 9y ago
      I used to believe that. I don't any longer. Your second paragraph basically agrees with me that bash affords bad code that ignores errors. I think it's past time for the programming world to take that seriously. We've had 60-some years of programmers insisting they can heroically just remember all the things they need to do to write correct code. And they are observably wrong. To disagree with me on that is basically to assert that software is nearly universally of a high quality and generally robust to all reasonably forseeable error conditions. Shell goes even beyond that, though; as evidenced by articles like this you can't even get good agreement on how you should write safe shell. When even experts can't agree on what's safe, that's just something unsuitable to any serious task.