7 ms·

I have to take exception to some of this; it's "technically correct" but like much of shell, likely full of terrifying edge cases that are best avoided. For in

by cduzz 3y ago

I have to take exception to some of this; it's "technically correct" but like much of shell, likely full of terrifying edge cases that are best avoided. For instance, using eval to have variables with variable names is madness. $ var="world" $ eval "hello_$var=value" $ eval printf '%s\n' "\$hello_$var" I suppose the fancy business with printf is flexing, but I suspect the majority of people scanning documents like this would prefer to have examples with echo "${variable%%string_modifier}" Escape sequences are mostly dependent on you being on a VT102 compatible terminal; I'm not sure what happens if you're on an ASR33 and you send this to it... I'd consider type checking (is this a float) to be similarly cursed witchcraft -- after all I've run into all sorts of "floats" that aren't just 123.456 strings... Overall -- there's a huge range of things shell can do that you probably should just avoid. Similarly, even if you can do it in a clever shell way (the bit shifting math for instance, or extensive string manipulation) should likely be done with external tools just because people understand "oh, awk or sed -- I know this!" instead of "what the hell is this line noise?" If you're writing performance optimized shell, well, probably put the keyboard down and walk away for a bit and reconsider your life choices. Good doc; I'll probably stick to the man page though.

Linux-Fan 3y ago
> I suppose the fancy business with printf is flexing, but I suspect the majority of people scanning documents like this would prefer to have examples with echo "${variable%%string_modifier}" POSIX itself contains some interesting notes about `echo` and `printf`, e.g. from the `printf` page: "The printf utility was added to provide functionality that has historically been provided by echo. However, due to irreconcilable differences in the various versions of echo extant, the version has few special features, leaving those to this new printf utility, which is based on one in the Ninth Edition system." The behaviour of `echo` is especially inconsistent for escape sequences. Often, `echo -e` is used to enable escape sequences for it but this is not POSIX compliant. For any "standards-perferring" shell scripting pages I'd thus think `printf` is the right way to present it if only to advertise that this builtin exists. > should likely be done with external tools just because people understand "oh, awk or sed -- I know this!" instead of "what the hell is this line noise?" I am not actually sure about this "I know this" part. I think many users of AWK only ever use it to do some sort of "access the n-th column of input" as in `echo a b c | awk '{ print $2 }'`. Similar for `sed` which is often only known and used for "replace string a by string b in input" use cases. > If you're writing performance optimized shell, well, probably put the keyboard down There is some difference between performance optimized and "hugely inefficient" that can make the difference between a task taking minutes vs. a few seconds. It may not seem much but shell scripts are often integrated into automated processes (think of build environments or system startup tasks) and there, seconds quickly accumulate. Whether this is relevant to your use case, only you can know. I found this "Pure Sh Bible" to be highly interesting, but there are some places where it has some rough edges and it may not show the "best practices" for standard use cases. Still a very useful resource.
- palunon 3y ago
  > Similar for `sed` which is often only known and used for "replace string a by string b in input" use cases. Sed has at least the vi users who may know ex command from the vi command mode. Awk needs to be learned for itself.
- cduzz 3y ago
  There are a huge variety of things you can do in shell that you simply shouldn't. If you're adventuring into a world where it matters if there's a newline at the end of your output as a result of your script running on noodlebsd or ache or freeGum, you've already lost. Don't play the game. Don't do things where it matters what the exact output of echo gives you. Did you know that you can't put a null into a shell variable? You can't. Also, you shouldn't ever be in a position to care. Did you know that you can fiddle with the "IFS" to let you parse CSVs? Don't ever do that; it works but nobody will ever understand what the hell you did. You can make an case statement and evaluate it with eval to make your own shell script that writes its own shell scripts. Please don't. All of these adventures are possible, and work fine, and I've done them and come back to the code a decade later and I both understand what I was trying to do and the code still works across a variety of bourne shell interpreters. Nevertheless, these are things that shouldn't be done except to flex.
ndsipa_pomu 3y ago
It's well worth learning to use printf instead of echo to avoid all the various ways that echo can break or is not well defined. https://mywiki.wooledge.org/BashPitfalls#echo_.24foo https://mywiki.wooledge.org/BashPitfalls#echo_.24foo
- koolba 3y ago
  The %q format alone justifies learning about printf. Vanilla echo is for chumps.
  - hddqsb 3y ago
    Note that this is supported by bash/zsh/ksh but not by POSIX sh.
- hddqsb 3y ago
  Yes. printf is not a "flex", it's the only safe way to display arbitrary strings. The link mentions one good reason to avoid echo (string might be "-n"), but another important reason is that the behaviour of echo is implementation-defined if the string contains backslashes (e.g. `echo 'foo\nbar'` displays one line in bash but two lines in dash and zsh).
chasil 3y ago
Here are my scary evals to use basic ANSI/vt220 color. N="$(printf '\033[')" x=30 for a in Bl R G Y B M C W # 4-bit Black Red Green Yellow Blue Magenta Cyan White do eval $a='$N'"'"$(( x))"m'" \ b$a='$N'"'"$((60+x))"m'" \ ${a}bg='$N'"'"$((10+x))"m'" \ b${a}bg='$N'"'"$((70+x))"m'" # bX=bright Xbg=background bXbg=brgt bgnd x=$((x+1)) done N="${N}0m" printf "$Y${Gbg}I am yellow on green.$N\n" I pasted this into my Android terminal, and it works as advertised.
- fsckboy 3y ago
  I am curious yellow on green is all that prints out? What's the rest of it for?
  - chasil 3y ago
    It's a general set of assignments that let you make any colors that you like. That's only an example. Use them all, if you want.
throwawaaarrgh 3y ago
Actually, eval, much like global variables (which are maligned by "real programmers" everywhere) actually works fine for a janky shell script. Also, don't use echo, use printf. echo has different semantics and potential footguns depending on the platform.
- jethro_tell 3y ago
  Isn't the problem with eval that you are running whatever is in the var? Probably variable expansion but maybe not and that could be not ideal
  - throwawaaarrgh 3y ago
    Programmers do that without realizing it all the time. There's like five classes of exploits that are just programmers assuming whatever variable they are interpolating is safe because they assume their language handles all security problems for them.
1letterunixname 3y ago
I'm scratching my head at most of it and there's large portions that are incomplete or trivial. bash is ubiquitous. If you're trying to develop an entire web framework to run on a Bourne shell from busybox, dash, or FreeBSD 4, there's probably a better way to do it. Use the tool you have, not a hypothetical one compatible with 1983. YAGNI. Here's a common pattern for appending to PATH system-wide once: # /etc/profile.d/foo.sh DIR=/opt/foo/bin case ":$PATH:" in *":$DIR:"*) ;; *) PATH="$PATH:$DIR"; export PATH ;; esac unset DIR If zsh or bash is available: # $1 var # $2 prepend this string to var # $3 separator (: default if omitted) # doesn't remove or prevent duplicates prepend() { eval "export $1=$2\${$1:+${3-:}\$$1}" } # $1 var # $2 append this string to var # $3 separator (: default if omitted) # doesn't remove or prevent duplicates append() { eval "export $1=\${$1:+\$$1${3-:}}$2" }
- hddqsb 3y ago
  Your "zsh/bash" code doesn't actually use any zsh/bash-only features, so it is compatible with POSIX sh. But you have a quoting bug: you should escape the `$` in `$2` and `${3...}` to protect from double-eval. I'd personally try to minimise the use of eval, e.g. prepend() { local val eval "val=\$$1" export "$1=$2${val:+${3-:}$val}" } append() { local val eval "val=\$$1" export "$1=${val:+$val${3-:}}$2" }
  - hddqsb 3y ago
    (`local` is not in POSIX, I know)
singron 3y ago
In bash, you can use `declare -n`. E.g. var=world declare -n tmp="hello_$var" tmp=value printf '%s\n' "$hello_world" # value I highly recommend this pattern if you want to emulate jagged 2d arrays or other nested data structures in bash for some reason.

by cduzz 3y ago

Linux-Fan 3y ago

> I suppose the fancy business with printf is flexing, but I suspect the majority of people scanning documents like this would prefer to have examples with echo "${variable%%string_modifier}" POSIX itself contains some interesting notes about `echo` and `printf`, e.g. from the `printf` page: "The printf utility was added to provide functionality that has historically been provided by echo. However, due to irreconcilable differences in the various versions of echo extant, the version has few special features, leaving those to this new printf utility, which is based on one in the Ninth Edition system." The behaviour of `echo` is especially inconsistent for escape sequences. Often, `echo -e` is used to enable escape sequences for it but this is not POSIX compliant. For any "standards-perferring" shell scripting pages I'd thus think `printf` is the right way to present it if only to advertise that this builtin exists. > should likely be done with external tools just because people understand "oh, awk or sed -- I know this!" instead of "what the hell is this line noise?" I am not actually sure about this "I know this" part. I think many users of AWK only ever use it to do some sort of "access the n-th column of input" as in `echo a b c | awk '{ print $2 }'`. Similar for `sed` which is often only known and used for "replace string a by string b in input" use cases. > If you're writing performance optimized shell, well, probably put the keyboard down There is some difference between performance optimized and "hugely inefficient" that can make the difference between a task taking minutes vs. a few seconds. It may not seem much but shell scripts are often integrated into automated processes (think of build environments or system startup tasks) and there, seconds quickly accumulate. Whether this is relevant to your use case, only you can know. I found this "Pure Sh Bible" to be highly interesting, but there are some places where it has some rough edges and it may not show the "best practices" for standard use cases. Still a very useful resource.

palunon 3y ago
> Similar for `sed` which is often only known and used for "replace string a by string b in input" use cases. Sed has at least the vi users who may know ex command from the vi command mode. Awk needs to be learned for itself.
cduzz 3y ago
There are a huge variety of things you can do in shell that you simply shouldn't. If you're adventuring into a world where it matters if there's a newline at the end of your output as a result of your script running on noodlebsd or ache or freeGum, you've already lost. Don't play the game. Don't do things where it matters what the exact output of echo gives you. Did you know that you can't put a null into a shell variable? You can't. Also, you shouldn't ever be in a position to care. Did you know that you can fiddle with the "IFS" to let you parse CSVs? Don't ever do that; it works but nobody will ever understand what the hell you did. You can make an case statement and evaluate it with eval to make your own shell script that writes its own shell scripts. Please don't. All of these adventures are possible, and work fine, and I've done them and come back to the code a decade later and I both understand what I was trying to do and the code still works across a variety of bourne shell interpreters. Nevertheless, these are things that shouldn't be done except to flex.

ndsipa_pomu 3y ago

It's well worth learning to use printf instead of echo to avoid all the various ways that echo can break or is not well defined. https://mywiki.wooledge.org/BashPitfalls#echo_.24foo https://mywiki.wooledge.org/BashPitfalls#echo_.24foo

koolba 3y ago
The %q format alone justifies learning about printf. Vanilla echo is for chumps.
- hddqsb 3y ago
  Note that this is supported by bash/zsh/ksh but not by POSIX sh.
hddqsb 3y ago
Yes. printf is not a "flex", it's the only safe way to display arbitrary strings. The link mentions one good reason to avoid echo (string might be "-n"), but another important reason is that the behaviour of echo is implementation-defined if the string contains backslashes (e.g. `echo 'foo\nbar'` displays one line in bash but two lines in dash and zsh).

chasil 3y ago

Here are my scary evals to use basic ANSI/vt220 color. N="$(printf '\033[')" x=30 for a in Bl R G Y B M C W # 4-bit Black Red Green Yellow Blue Magenta Cyan White do eval $a='$N'"'"$(( x))"m'" \ b$a='$N'"'"$((60+x))"m'" \ ${a}bg='$N'"'"$((10+x))"m'" \ b${a}bg='$N'"'"$((70+x))"m'" # bX=bright Xbg=background bXbg=brgt bgnd x=$((x+1)) done N="${N}0m" printf "$Y${Gbg}I am yellow on green.$N\n" I pasted this into my Android terminal, and it works as advertised.

fsckboy 3y ago
I am curious yellow on green is all that prints out? What's the rest of it for?
- chasil 3y ago
  It's a general set of assignments that let you make any colors that you like. That's only an example. Use them all, if you want.

throwawaaarrgh 3y ago

Actually, eval, much like global variables (which are maligned by "real programmers" everywhere) actually works fine for a janky shell script. Also, don't use echo, use printf. echo has different semantics and potential footguns depending on the platform.

jethro_tell 3y ago
Isn't the problem with eval that you are running whatever is in the var? Probably variable expansion but maybe not and that could be not ideal
- throwawaaarrgh 3y ago
  Programmers do that without realizing it all the time. There's like five classes of exploits that are just programmers assuming whatever variable they are interpolating is safe because they assume their language handles all security problems for them.

1letterunixname 3y ago

I'm scratching my head at most of it and there's large portions that are incomplete or trivial. bash is ubiquitous. If you're trying to develop an entire web framework to run on a Bourne shell from busybox, dash, or FreeBSD 4, there's probably a better way to do it. Use the tool you have, not a hypothetical one compatible with 1983. YAGNI. Here's a common pattern for appending to PATH system-wide once: # /etc/profile.d/foo.sh DIR=/opt/foo/bin case ":$PATH:" in *":$DIR:"*) ;; *) PATH="$PATH:$DIR"; export PATH ;; esac unset DIR If zsh or bash is available: # $1 var # $2 prepend this string to var # $3 separator (: default if omitted) # doesn't remove or prevent duplicates prepend() { eval "export $1=$2\${$1:+${3-:}\$$1}" } # $1 var # $2 append this string to var # $3 separator (: default if omitted) # doesn't remove or prevent duplicates append() { eval "export $1=\${$1:+\$$1${3-:}}$2" }

hddqsb 3y ago
Your "zsh/bash" code doesn't actually use any zsh/bash-only features, so it is compatible with POSIX sh. But you have a quoting bug: you should escape the `$` in `$2` and `${3...}` to protect from double-eval. I'd personally try to minimise the use of eval, e.g. prepend() { local val eval "val=\$$1" export "$1=$2${val:+${3-:}$val}" } append() { local val eval "val=\$$1" export "$1=${val:+$val${3-:}}$2" }
- hddqsb 3y ago
  (`local` is not in POSIX, I know)

singron 3y ago

In bash, you can use `declare -n`. E.g. var=world declare -n tmp="hello_$var" tmp=value printf '%s\n' "$hello_world" # value I highly recommend this pattern if you want to emulate jagged 2d arrays or other nested data structures in bash for some reason.