4 ms·

The thought of a Bash framework scares me. Next thing you know someone will write React in it... Just freakin learn perl or python! Do not let this beast grow

by tomc1985 8y ago

The thought of a Bash framework scares me. Next thing you know someone will write React in it... Just freakin learn perl or python! Do not let this beast grow any larger.

Annatar 8y ago
Both Perl and Python are way more complicated than shell programming, and in all my decades of doing so on the command line, I’ve yet to find a problem which a full blown shell + AWK couldn’t solve. And that combination is still simpler to program in than Python or Perl. Had you grown up on a real UNIX, you’d have never written what you wrote.
- kamaal 8y ago
  Perl is far more than anything you can ever do in Bash, even the basic things. Then there is also the famous Tom Christiansen Essay on C-shell: https://www-uxsup.csx.cam.ac.uk/misc/csh.html https://www-uxsup.csx.cam.ac.uk/misc/csh.html You can even do Lispy things in Perl. You can do functional stuff: https://hop.perl.plover.com/ https://hop.perl.plover.com/ You can do unicode regexes in Perl: https://perldoc.perl.org/perlunicode.html https://perldoc.perl.org/perlunicode.html You can do large file processing in Perl: https://www.perlmonks.org/?node_id=344087 https://www.perlmonks.org/?node_id=344087 , https://www.perlmonks.org/?node_id=1118102 https://www.perlmonks.org/?node_id=1118102 Perl's file and string manipulation capabilities has no match. If you use awk you have to think in the line paradigm. Perl gives you not just that but a lot more. Perl regexes are by far the strongest of their kinds programming language out there(first class entities). In fact the whole reason why Larry Wall invented Perl is at some point in time you max out what you can do with things like awk and sed.
  - Annatar 8y ago
    I spent 3.5 years debugging and maintaining Perl code. For a living. I have formal education in programming applications in Perl. And I stand by what I wrote about shell + AWK, especially AWK, any day of the week. By the by, I can do unlimited number of things in a shell program, things no other programming language can do, because I can call any other program from it. Apart from assembler, shell is the second most powerful tool because of that characteristic.
    - kamaal 8y ago
      Sure you can curl a data end point and then parse the resulting JSON/XML using regexes, cut, tr in bash+awk+sed, but your code will break even on a slight change of output rendered. These sort of things are just the beginning. People will be surprised how hard it is to parse something like a csv. You start to begin discovering limits when you start doing things like error handling. Writing slightly complicated regexes, or if you need a little complicated code written relying on if/for more often. More everyday use cases that come to my mind are use of things like Data::Dumper, qw, open/while<>/close paradigm code, arrays, hashmaps, grepping over large lists, unicode, heredocs, handling binary data etc. Nothing to take away from Bash. But its really to glue together a small bunch of unix commands in progression. If you are doing anything more than a 100 line program, you are better off with Perl.
      - Annatar 8y ago
        “but your code will break even on a slight change of output rendered.” I see you still don’t get it. I wouldn’t as an experienced shell programmer construct a JSON/XML parser; I use libxslt for XML and I compiled jq for parsing JSON. It’s the UNIX way. The Zen of this eludes you still. Think deeper.
        kamaal 8y ago
        >>The Zen of this eludes you still. Seems like a blessing in disguise in this case, if such zen exists. I should use shell+awk+sed+libxslt+jq and all that, instead of using Perl + XML::Simple?
        Annatar 8y ago
        Indeed, for that XML::Simple would be an operational nightmare to deploy and maintain (being fused into Perl’s tree), and your code would be unreadable and unmaintainable by anyone else, for such is the nature of Perl. And then there is the UNIX way, with both jq and libxslt being reusable for other data without having to write a dedicated program. sed by the way is unnecessary if one has mastered AWK. There are too many things you have not even begun to consider yet. You could start with taking those developer-convenience glasses off, and putting the system engineer glasses on.
- tomc1985 8y ago
  Perl sits in a happy medium between bash and full-on programming. Yes bash makes piping really easy but IMO that is about it for advantages.
chriswarbo 8y ago
> Just freakin learn perl or python! For calculations? Sure; but that's not really what bash is for. Shells are excellent at invoking and managing subprocesses and piping. Python (not sure about Perl, I've not really used it) is terrible at those things. Take a simple, very common piece of bash code like `foo | bar`. How might we do this in Python? Maybe we reach for the builtin `subprocess` module: import subprocess foo_output = subprocess.check_call(['foo']) bar = subprocess.Popen(['bar'], stdin=subprocess.PIPE) bar.communicate(foo_output) Except that this isn't a pipe: it will run `foo` to completion, storing all of the output in memory, then call `bar` on this data, e.g. `foo_output=$(foo); echo "$foo_output" | bar`. This is unsuitable for long-lived processes (e.g. if `foo` is long lived and `bar` is meant to be logging its output), or if there is a lot of data (e.g. `foo` is generating GBs of text and `bar` is summarising it, like `wc -l`). OK, maybe instead of `check_call` (which is blocking) we make `foo` asynchronous. Note that we can't use `foo.communicate` to get its output, since that would also block. What if we just shuttle data between the two manually, line by line (urgh)? import subprocess foo = subprocess.Popen(['foo'], stdout=subprocess.PIPE) bar = subprocess.Popen(['bar'], stdin=subprocess.PIPE) while foo.poll() is None: bar.stdin.write(foo.stdout.readline()) bar.stdin.close() bar.wait() This appears to work, especially when testing with small amounts of data. Yet it's actually a timebomb, since it will deadlock when the subprocesses' pipe buffers fill up (this is why the documentation tells us to use `communicate`, except that we can't since that's blocking https://docs.python.org/3.7/library/subprocess.html https://docs.python.org/3.7/library/subprocess.html ). It's at this point that we move the data shuttling into a separate thread: import subprocess import threading foo = subprocess.Popen(['foo'], stdout=subprocess.PIPE) bar = subprocess.Popen(['bar'], stdin=subprocess.PIPE) def shuttle(): while foo.poll() is not None: bar.stdin.write(foo.stdout.readline()) foo.stdout.close() bar.stdin.close() thread = threading.Thread(target=shuttle) thread.daemon = True thread.start() thread.join() bar.wait() Now we've got a pile of code mixing multiprocessing with multithreading, in a domain known to have deadlocks, with hand-written hard-coded line buffering. At this point, I'd say just freakin learn bash!
- kamaal 8y ago
  >>not sure about Perl, I've not really used it Perl is great at these things. In fact you talk shell native tongue in Perl. use strict; use warnings; my @output = `your_command | your_another_command`; foreach my $line (@output) { chomp; #removes new line $line = $_; if ($line =~ /your regex goes here/) { #if match, use $1, $2... to get the groups matched #your buisness logic goes here } }

The thought of a Bash framework scares me. Next thing you know someone will write React in it... Just freakin learn perl or python! Do not let this beast grow any larger.

Annatar 8y ago

Both Perl and Python are way more complicated than shell programming, and in all my decades of doing so on the command line, I’ve yet to find a problem which a full blown shell + AWK couldn’t solve. And that combination is still simpler to program in than Python or Perl. Had you grown up on a real UNIX, you’d have never written what you wrote.

kamaal 8y ago
Perl is far more than anything you can ever do in Bash, even the basic things. Then there is also the famous Tom Christiansen Essay on C-shell: https://www-uxsup.csx.cam.ac.uk/misc/csh.html https://www-uxsup.csx.cam.ac.uk/misc/csh.html You can even do Lispy things in Perl. You can do functional stuff: https://hop.perl.plover.com/ https://hop.perl.plover.com/ You can do unicode regexes in Perl: https://perldoc.perl.org/perlunicode.html https://perldoc.perl.org/perlunicode.html You can do large file processing in Perl: https://www.perlmonks.org/?node_id=344087 https://www.perlmonks.org/?node_id=344087 , https://www.perlmonks.org/?node_id=1118102 https://www.perlmonks.org/?node_id=1118102 Perl's file and string manipulation capabilities has no match. If you use awk you have to think in the line paradigm. Perl gives you not just that but a lot more. Perl regexes are by far the strongest of their kinds programming language out there(first class entities). In fact the whole reason why Larry Wall invented Perl is at some point in time you max out what you can do with things like awk and sed.
- Annatar 8y ago
  I spent 3.5 years debugging and maintaining Perl code. For a living. I have formal education in programming applications in Perl. And I stand by what I wrote about shell + AWK, especially AWK, any day of the week. By the by, I can do unlimited number of things in a shell program, things no other programming language can do, because I can call any other program from it. Apart from assembler, shell is the second most powerful tool because of that characteristic.
  - kamaal 8y ago
    Sure you can curl a data end point and then parse the resulting JSON/XML using regexes, cut, tr in bash+awk+sed, but your code will break even on a slight change of output rendered. These sort of things are just the beginning. People will be surprised how hard it is to parse something like a csv. You start to begin discovering limits when you start doing things like error handling. Writing slightly complicated regexes, or if you need a little complicated code written relying on if/for more often. More everyday use cases that come to my mind are use of things like Data::Dumper, qw, open/while<>/close paradigm code, arrays, hashmaps, grepping over large lists, unicode, heredocs, handling binary data etc. Nothing to take away from Bash. But its really to glue together a small bunch of unix commands in progression. If you are doing anything more than a 100 line program, you are better off with Perl.
    - Annatar 8y ago
      “but your code will break even on a slight change of output rendered.” I see you still don’t get it. I wouldn’t as an experienced shell programmer construct a JSON/XML parser; I use libxslt for XML and I compiled jq for parsing JSON. It’s the UNIX way. The Zen of this eludes you still. Think deeper.
      - kamaal 8y ago
        >>The Zen of this eludes you still. Seems like a blessing in disguise in this case, if such zen exists. I should use shell+awk+sed+libxslt+jq and all that, instead of using Perl + XML::Simple?
        Annatar 8y ago
        Indeed, for that XML::Simple would be an operational nightmare to deploy and maintain (being fused into Perl’s tree), and your code would be unreadable and unmaintainable by anyone else, for such is the nature of Perl. And then there is the UNIX way, with both jq and libxslt being reusable for other data without having to write a dedicated program. sed by the way is unnecessary if one has mastered AWK. There are too many things you have not even begun to consider yet. You could start with taking those developer-convenience glasses off, and putting the system engineer glasses on.
tomc1985 8y ago
Perl sits in a happy medium between bash and full-on programming. Yes bash makes piping really easy but IMO that is about it for advantages.

chriswarbo 8y ago

> Just freakin learn perl or python! For calculations? Sure; but that's not really what bash is for. Shells are excellent at invoking and managing subprocesses and piping. Python (not sure about Perl, I've not really used it) is terrible at those things. Take a simple, very common piece of bash code like `foo | bar`. How might we do this in Python? Maybe we reach for the builtin `subprocess` module: import subprocess foo_output = subprocess.check_call(['foo']) bar = subprocess.Popen(['bar'], stdin=subprocess.PIPE) bar.communicate(foo_output) Except that this isn't a pipe: it will run `foo` to completion, storing all of the output in memory, then call `bar` on this data, e.g. `foo_output=$(foo); echo "$foo_output" | bar`. This is unsuitable for long-lived processes (e.g. if `foo` is long lived and `bar` is meant to be logging its output), or if there is a lot of data (e.g. `foo` is generating GBs of text and `bar` is summarising it, like `wc -l`). OK, maybe instead of `check_call` (which is blocking) we make `foo` asynchronous. Note that we can't use `foo.communicate` to get its output, since that would also block. What if we just shuttle data between the two manually, line by line (urgh)? import subprocess foo = subprocess.Popen(['foo'], stdout=subprocess.PIPE) bar = subprocess.Popen(['bar'], stdin=subprocess.PIPE) while foo.poll() is None: bar.stdin.write(foo.stdout.readline()) bar.stdin.close() bar.wait() This appears to work, especially when testing with small amounts of data. Yet it's actually a timebomb, since it will deadlock when the subprocesses' pipe buffers fill up (this is why the documentation tells us to use `communicate`, except that we can't since that's blocking https://docs.python.org/3.7/library/subprocess.html https://docs.python.org/3.7/library/subprocess.html ). It's at this point that we move the data shuttling into a separate thread: import subprocess import threading foo = subprocess.Popen(['foo'], stdout=subprocess.PIPE) bar = subprocess.Popen(['bar'], stdin=subprocess.PIPE) def shuttle(): while foo.poll() is not None: bar.stdin.write(foo.stdout.readline()) foo.stdout.close() bar.stdin.close() thread = threading.Thread(target=shuttle) thread.daemon = True thread.start() thread.join() bar.wait() Now we've got a pile of code mixing multiprocessing with multithreading, in a domain known to have deadlocks, with hand-written hard-coded line buffering. At this point, I'd say just freakin learn bash!

kamaal 8y ago
>>not sure about Perl, I've not really used it Perl is great at these things. In fact you talk shell native tongue in Perl. use strict; use warnings; my @output = `your_command | your_another_command`; foreach my $line (@output) { chomp; #removes new line $line = $_; if ($line =~ /your regex goes here/) { #if match, use $1, $2... to get the groups matched #your buisness logic goes here } }