3 ms·

I will just leave this here. "I am a strong believer that Bourne-derived languages are extremely bad, on the same order of badness as Perl, for programming, an

by chasil 5y ago

I will just leave this here. "I am a strong believer that Bourne-derived languages are extremely bad, on the same order of badness as Perl, for programming, and consider programming sh for any purpose other than as a super-portable, lowest-common-denominator platform for build or bootstrap scripts and the like, as an extremely misguided endeavor. As such you won’t see me spending many words on extensions particular to ksh, Bash, or whatever other shells may be popular." http://www.etalabs.net/sh_tricks.html http://www.etalabs.net/sh_tricks.html

marginalia_nu 5y ago
I think it's easy to sort of gloss over how good shell scripts are at the sort of things they actually are good at. Consider this: for f in *png; do convert $f $(echo $f | sed 's/png$/webp/'); done This is like at least a screen of code in in most languages, you get that sinking feeling where you need to fork a process and deal with its lifecycle and deal with some standard library to list files and don't forget to close the file handle for the directory and god forgive if you run a program with actual output then you need to read from a pipe. Like, geesh, do I have to write a whole program to convert a few files? In a shell, it's just something you type up because you needed to convert a few files. Don't get me wrong, shell scripts are extremely bad at most things involving non-trivial branches, long-lived state, really most things other languages are good at, but man are they ever good at some things most programming languages are bad at.
- leephillips 5y ago
  lee@sp3tmp1:ls *.png 'a b.png' c.png lee@sp3tmp1:for f in *png; do convert $f $(echo $f | sed 's/png$/web'); done convert-im6.q16: unable to open image `a': No such file or directory @ error/blob.c/OpenBlob/2924. convert-im6.q16: no decode delegate for this image format `' @ error/constitute.c/ReadImage/575. convert-im6.q16: unable to open image `b.png': No such file or directory @ error/blob.c/OpenBlob/2924. convert-im6.q16: unable to open image `b.png': No such file or directory @ error/blob.c/OpenBlob/2924. convert-im6.q16: no decode delegate for this image format `' @ error/constitute.c/ReadImage/575. convert-im6.q16: no images defined `b.webp' @ error/convert.c/ConvertImageCommand/3229. [etc.] I think you could be making a more complicated point.
- _kst_ 5y ago
  Better: for f in *.png; do convert "$f" "${f/%.png/.webp}" ; done Quoting the arguments lets it correctly handle file names with funny characters, including spaces. Using the `${parameter/pattern/string}` parameter substitution means the shell does all the work itself without invoking an external command (other than convert, of course). The '%' in the parameter substitution requires the pattern to match at the end, so "foo.png.png" becomes "foo.png.webp" and not "foo.webp.png" -- an unlikely corner case, but you might as well do it right. ("foo.png.blah" doesn't match "*.png" in the first place.) For elaborate commands like this, I typically run it once with an "echo" inserted ("echo convert ..."), confirm that the resulting commands make sense, and then hit up-arrow to edit the command and delete the echo. https://www.gnu.org/software/bash/manual/html_node/Shell-Parameter-Expansion.html https://www.gnu.org/software/bash/manual/html_node/Shell-Par...
  - Flimm 5y ago
    Quoting is not enough, as that won't handle filenames beginning with dash correctly.
- chasil 5y ago
  Your script fails if the files referenced by $f contain white spaces or a newline. You should script defensively, for when this is run on Windows or elsewhere [maybe Android] that will break this assumption. Newlines can also be part of a filename. For extreme rigor, use find -print0 and xargs -0. I have covered some of this here: https://www.linuxjournal.com/content/parallel-shells-xargs-utilize-all-your-cpu-cores-unix-and-windows https://www.linuxjournal.com/content/parallel-shells-xargs-u... (Somebody took me to the Hacker News front page for this, and nobody said that I made a mistake.)
  - Flimm 5y ago
    I agree. It also fails if the filename starts with a dash.
  - marginalia_nu 5y ago
    > You should script defensively, for when this is run on Windows or elsewhere [maybe Android] that will break this assumption. I think you are missing the point. I wouldn't run a shellscript on Windows or any other place than the terminal I entered it into. It's an expression of something I want to do at that particular time in that particular directory. The entire point of my post is that if I wished to write something portable and robust, I wouldn't use bash in the first place. That's not what bash is good at. But quick one-offs, holy crap is it good at that.
    - chasil 5y ago
      I've run many Borne/POSIX shell scripts on Windows for things that I've needed to do. The most straightforward POSIX shell on Windows is the Busybox port, which uses the Almquist shell (ash) with many bashisms reintroduced. https://frippery.org/busybox/ https://frippery.org/busybox/
  - kwhitefoot 5y ago
    > fails if the files referenced by $f contain white spaces or a newline. This is said every time Bash scripting is mentioned. Why people can't get into the habit of quoting all variables in Bash is beyond me and the mention of white space makes sense; but how often, if ever, have you encountered a file name that contained a newline? Most of the Unix tools are line oriented by default anyway and are frequently used to manipulate lists of file names and paths.
- pizza234 5y ago
  >> I think it's easy to sort of gloss over how good shell scripts are at the sort of things they actually are good at. >> Consider this: > for f in *png; do convert $f $(echo $f | sed 's/png$/webp/'); done This is buggy, and it actually represents exactly the problem with Bash scripting.

by chasil 5y ago

marginalia_nu 5y ago

I think it's easy to sort of gloss over how good shell scripts are at the sort of things they actually are good at. Consider this: for f in *png; do convert $f $(echo $f | sed 's/png$/webp/'); done This is like at least a screen of code in in most languages, you get that sinking feeling where you need to fork a process and deal with its lifecycle and deal with some standard library to list files and don't forget to close the file handle for the directory and god forgive if you run a program with actual output then you need to read from a pipe. Like, geesh, do I have to write a whole program to convert a few files? In a shell, it's just something you type up because you needed to convert a few files. Don't get me wrong, shell scripts are extremely bad at most things involving non-trivial branches, long-lived state, really most things other languages are good at, but man are they ever good at some things most programming languages are bad at.

leephillips 5y ago
lee@sp3tmp1:ls *.png 'a b.png' c.png lee@sp3tmp1:for f in *png; do convert $f $(echo $f | sed 's/png$/web'); done convert-im6.q16: unable to open image `a': No such file or directory @ error/blob.c/OpenBlob/2924. convert-im6.q16: no decode delegate for this image format `' @ error/constitute.c/ReadImage/575. convert-im6.q16: unable to open image `b.png': No such file or directory @ error/blob.c/OpenBlob/2924. convert-im6.q16: unable to open image `b.png': No such file or directory @ error/blob.c/OpenBlob/2924. convert-im6.q16: no decode delegate for this image format `' @ error/constitute.c/ReadImage/575. convert-im6.q16: no images defined `b.webp' @ error/convert.c/ConvertImageCommand/3229. [etc.] I think you could be making a more complicated point.
_kst_ 5y ago
Better: for f in *.png; do convert "$f" "${f/%.png/.webp}" ; done Quoting the arguments lets it correctly handle file names with funny characters, including spaces. Using the `${parameter/pattern/string}` parameter substitution means the shell does all the work itself without invoking an external command (other than convert, of course). The '%' in the parameter substitution requires the pattern to match at the end, so "foo.png.png" becomes "foo.png.webp" and not "foo.webp.png" -- an unlikely corner case, but you might as well do it right. ("foo.png.blah" doesn't match "*.png" in the first place.) For elaborate commands like this, I typically run it once with an "echo" inserted ("echo convert ..."), confirm that the resulting commands make sense, and then hit up-arrow to edit the command and delete the echo. https://www.gnu.org/software/bash/manual/html_node/Shell-Parameter-Expansion.html https://www.gnu.org/software/bash/manual/html_node/Shell-Par...
- Flimm 5y ago
  Quoting is not enough, as that won't handle filenames beginning with dash correctly.
chasil 5y ago
Your script fails if the files referenced by $f contain white spaces or a newline. You should script defensively, for when this is run on Windows or elsewhere [maybe Android] that will break this assumption. Newlines can also be part of a filename. For extreme rigor, use find -print0 and xargs -0. I have covered some of this here: https://www.linuxjournal.com/content/parallel-shells-xargs-utilize-all-your-cpu-cores-unix-and-windows https://www.linuxjournal.com/content/parallel-shells-xargs-u... (Somebody took me to the Hacker News front page for this, and nobody said that I made a mistake.)
- Flimm 5y ago
  I agree. It also fails if the filename starts with a dash.
- marginalia_nu 5y ago
  > You should script defensively, for when this is run on Windows or elsewhere [maybe Android] that will break this assumption. I think you are missing the point. I wouldn't run a shellscript on Windows or any other place than the terminal I entered it into. It's an expression of something I want to do at that particular time in that particular directory. The entire point of my post is that if I wished to write something portable and robust, I wouldn't use bash in the first place. That's not what bash is good at. But quick one-offs, holy crap is it good at that.
  - chasil 5y ago
    I've run many Borne/POSIX shell scripts on Windows for things that I've needed to do. The most straightforward POSIX shell on Windows is the Busybox port, which uses the Almquist shell (ash) with many bashisms reintroduced. https://frippery.org/busybox/ https://frippery.org/busybox/
- kwhitefoot 5y ago
  > fails if the files referenced by $f contain white spaces or a newline. This is said every time Bash scripting is mentioned. Why people can't get into the habit of quoting all variables in Bash is beyond me and the mention of white space makes sense; but how often, if ever, have you encountered a file name that contained a newline? Most of the Unix tools are line oriented by default anyway and are frequently used to manipulate lists of file names and paths.
pizza234 5y ago
>> I think it's easy to sort of gloss over how good shell scripts are at the sort of things they actually are good at. >> Consider this: > for f in *png; do convert $f $(echo $f | sed 's/png$/webp/'); done This is buggy, and it actually represents exactly the problem with Bash scripting.