5 ms·

For many things shell scripts are the easiest and most expressive way to write them. And probably you don't know that human resources are more expensive than cp

For many things shell scripts are the easiest and most expressive way to write them. And probably you don't know that human resources are more expensive than cpu cycles. Sure, I love python, but if I want to find and filter some files nothing is easier to write than bash, so it's possible to have a quite big bash script even doing a good job and being well readable. I'm honestly a little shocked that this doesn't seem to be common knowledge anymore and your post got upvoted so high. My main language is Python btw.

meddlepal 7y ago
Easiest and most expressive way for you maybe. And yea human resources are expensive... doubly so when you need to throw more of them at this thing written by a former developer that thought he or she was saving money for the org by not spending an extra 30 minutes writing the program in Python or Go. > but if I want to find and filter some files nothing is easier to write than bash That's like 10 lines in Python and much easier to read and test.
- F-0X 7y ago
  > That's like 10 lines in Python and much easier to read and test. It's a single command in bash, invoking a program far more efficient than anything you can put together in python. It is literally a tool written for this exact purpose. I think it is simply ridiculous to not use a pre-existing tool that frankly you cannot out-do with a scripting language. Even if you are writing a python script, you should _still_ use find to search for files if there is _any_ complexity in the search requirements.
- narnianal 7y ago
  > Easiest and most expressive way for you maybe. Look. We are working an professional field with a strong scientific foundation. There is no "this bridge is harder to drive over for me therefore I think it's bad architecture". The easiness and expressiveness depend on the problem you try to solve, not on the people, their tastes or their experience. Shell tools are highly optimized to work in pipes, to work on files and to work on line separated entries like lines of code or lines of CSV. So if you have a problem where you need to pipe file handling into some filtering and into some line based transformations then you will have a hard time to find anything as easy or expressive as a bash script. And most projects on a unix-like operating system at least start like this: Understand the problem based on database dumps, logs, csv data measurements, previously written code etc. If your regular expressions are not able to cover all the edge cases anymore and/or become unreadably complex then of course you should start an actual program. That only happens a few weeks into most topics though. > doubly so when you need to throw more of them at this thing written by a former developer that thought he or she was saving money for the org by not spending an extra 30 minutes writing the program in Python or Go. Yes, there are people who lack the skill or ambition to develop good code. We all hate when we need to deal with their stuff. But honestly, would they have written better code if they had used Python or Java or C++ for that problem? Unlikely. A good coder would have some feeling for when he outsources some functionality to a python script. And if he develops a code base for a longer time most of his code will be in a real programming language. But even then it might be wrapped into a thin layer of bash to use find or grep or something that cleans up the input a little.
DonHopkins 7y ago
What if one of your file names has a space in it? Most bash scripts will crash when that happens. Please don't tell me never to use spaces in file names. How many human resources are required to track down and fix why a bash script suddenly starts failing mysteriously, years after it was written, when somebody mistakenly dares to use a space in a file name? That kind of problem never happens with Python scripts (including problems with file names including other punctuation and unicode characters), because Python simply represents file names as strings. For a scripting language that was indented to manipulate file names all the time, bash sure fucked up big time. https://www.linuxjournal.com/article/10954 https://www.linuxjournal.com/article/10954 >Work the Shell - Dealing with Spaces in Filenames >[...] I've outlined three possible solution paths herein: modifying the IFS value, ensuring that you always quote filenames where referenced, and rewriting filenames internally to replace spaces with unlikely character sequences, reversing it on your way out of the script. >By the way, have you ever tried using the find|xargs pair with filenames that have spaces? It's sufficiently complicated that modern versions of these two commands have special arguments to denote that spaces might appear as part of the filenames: find -print and xargs -0 (and typically, they're not the same flags, but that's another story). >During the years I've been writing this column, I've more than once tripped up on this particular problem and received e-mail messages from readers sharing how a sample script tosses up its bits when a file with a space in its name appears. They're right. >My defensive reaction is “dude, don't use spaces in filenames”, but that's not really a good long-term solution, is it? >What I'd like to do instead is open this up for discussion on the Linux Journal discussion boards: how do you solve this problem within your scripts? Or, do you just religiously avoid using spaces in your filenames? >Dave Taylor has been hacking shell scripts for a really long time, 30 years. He's the author of the popular Wicked Cool Shell Scripts and can be found on Twitter as @DaveTaylor and more generally at www.DaveTaylorOnline.com. (Dave wrote that 8 years ago, and there are still zero comments on it. Nobody has a good solution! Bash is just permanently fucked when it comes to file names.)
- pdkl95 7y ago
  bash has handled spaces in filenames easily for a long time: #!/bin/bash # run this scri0t with 0 or more filename args for file in "$@" ; do do_something_with "${filename}" done > Nobody has a good solution! Bash is just permanently fucked when it comes to file name From the man page bash(1): Special Parameters @ Expands to the positional parameters, starting from one. When the expansion occurs within double quotes, each parameter expands to a separate word. That is, "$@" is equivalent to "$1" "$2" ...
teknico 7y ago
Shell scripts and doing a good job don't go well together. For simpler, more expressive Python system scripts use Plumbum: https://plumbum.readthedocs.io/ https://plumbum.readthedocs.io/
- narnianal 7y ago
  Yeah and don't use vim for code editing. It's too much work to learn, right? Rather use nano. </sarcasm>

meddlepal 7y ago

Easiest and most expressive way for you maybe. And yea human resources are expensive... doubly so when you need to throw more of them at this thing written by a former developer that thought he or she was saving money for the org by not spending an extra 30 minutes writing the program in Python or Go. > but if I want to find and filter some files nothing is easier to write than bash That's like 10 lines in Python and much easier to read and test.

F-0X 7y ago
> That's like 10 lines in Python and much easier to read and test. It's a single command in bash, invoking a program far more efficient than anything you can put together in python. It is literally a tool written for this exact purpose. I think it is simply ridiculous to not use a pre-existing tool that frankly you cannot out-do with a scripting language. Even if you are writing a python script, you should _still_ use find to search for files if there is _any_ complexity in the search requirements.
narnianal 7y ago
> Easiest and most expressive way for you maybe. Look. We are working an professional field with a strong scientific foundation. There is no "this bridge is harder to drive over for me therefore I think it's bad architecture". The easiness and expressiveness depend on the problem you try to solve, not on the people, their tastes or their experience. Shell tools are highly optimized to work in pipes, to work on files and to work on line separated entries like lines of code or lines of CSV. So if you have a problem where you need to pipe file handling into some filtering and into some line based transformations then you will have a hard time to find anything as easy or expressive as a bash script. And most projects on a unix-like operating system at least start like this: Understand the problem based on database dumps, logs, csv data measurements, previously written code etc. If your regular expressions are not able to cover all the edge cases anymore and/or become unreadably complex then of course you should start an actual program. That only happens a few weeks into most topics though. > doubly so when you need to throw more of them at this thing written by a former developer that thought he or she was saving money for the org by not spending an extra 30 minutes writing the program in Python or Go. Yes, there are people who lack the skill or ambition to develop good code. We all hate when we need to deal with their stuff. But honestly, would they have written better code if they had used Python or Java or C++ for that problem? Unlikely. A good coder would have some feeling for when he outsources some functionality to a python script. And if he develops a code base for a longer time most of his code will be in a real programming language. But even then it might be wrapped into a thin layer of bash to use find or grep or something that cleans up the input a little.

DonHopkins 7y ago

What if one of your file names has a space in it? Most bash scripts will crash when that happens. Please don't tell me never to use spaces in file names. How many human resources are required to track down and fix why a bash script suddenly starts failing mysteriously, years after it was written, when somebody mistakenly dares to use a space in a file name? That kind of problem never happens with Python scripts (including problems with file names including other punctuation and unicode characters), because Python simply represents file names as strings. For a scripting language that was indented to manipulate file names all the time, bash sure fucked up big time. https://www.linuxjournal.com/article/10954 https://www.linuxjournal.com/article/10954 >Work the Shell - Dealing with Spaces in Filenames >[...] I've outlined three possible solution paths herein: modifying the IFS value, ensuring that you always quote filenames where referenced, and rewriting filenames internally to replace spaces with unlikely character sequences, reversing it on your way out of the script. >By the way, have you ever tried using the find|xargs pair with filenames that have spaces? It's sufficiently complicated that modern versions of these two commands have special arguments to denote that spaces might appear as part of the filenames: find -print and xargs -0 (and typically, they're not the same flags, but that's another story). >During the years I've been writing this column, I've more than once tripped up on this particular problem and received e-mail messages from readers sharing how a sample script tosses up its bits when a file with a space in its name appears. They're right. >My defensive reaction is “dude, don't use spaces in filenames”, but that's not really a good long-term solution, is it? >What I'd like to do instead is open this up for discussion on the Linux Journal discussion boards: how do you solve this problem within your scripts? Or, do you just religiously avoid using spaces in your filenames? >Dave Taylor has been hacking shell scripts for a really long time, 30 years. He's the author of the popular Wicked Cool Shell Scripts and can be found on Twitter as @DaveTaylor and more generally at www.DaveTaylorOnline.com. (Dave wrote that 8 years ago, and there are still zero comments on it. Nobody has a good solution! Bash is just permanently fucked when it comes to file names.)

pdkl95 7y ago
bash has handled spaces in filenames easily for a long time: #!/bin/bash # run this scri0t with 0 or more filename args for file in "$@" ; do do_something_with "${filename}" done > Nobody has a good solution! Bash is just permanently fucked when it comes to file name From the man page bash(1): Special Parameters @ Expands to the positional parameters, starting from one. When the expansion occurs within double quotes, each parameter expands to a separate word. That is, "$@" is equivalent to "$1" "$2" ...

teknico 7y ago

Shell scripts and doing a good job don't go well together. For simpler, more expressive Python system scripts use Plumbum: https://plumbum.readthedocs.io/ https://plumbum.readthedocs.io/

narnianal 7y ago
Yeah and don't use vim for code editing. It's too much work to learn, right? Rather use nano. </sarcasm>