3 ms·

I think it's important for anyone — developers especially — using a shell to understand basic shell principles. Shell is a REPL that has some special filesyste

I think it's important for anyone — developers especially — using a shell to understand basic shell principles. Shell is a REPL that has some special filesystem-related helpers. Looking through $PATH for any +x file that matches the name of the first word entered. Performing glob expansion (*, ?, {foo,bar}...) on all but the first word. Backticks. Environment variable scoping (difference between `export FOO=bar` and `FOO=bar ./my_program`). Presence of "." and ".." in the FS. And so forth. Once someone wraps their head around the execution model, they should be golden.

formerly_proven 5y ago
You can actually use expansion even in the first word, just not for searching $PATH because that'd be inefficient.
- necovek 5y ago
  You are right, for some reason I was focused on the combined search-through-$PATH and glob expansion at the same time.
numerobis 5y ago
Are backticks necessary these days? I thought it was advised to use `$(...)` instead…
- necovek 5y ago
  If you are going to accidentally type them in (maybe pasting some markdown?), it's surely good to know what they do.
darrenf 5y ago
If we're talking about bash, I think it's a mistake to refer to any of this as "glob expansion", a term which doesn't appear in the bash man page. Rather, it differentiates between multiple types of expansion and the order in which they take place. Quote: The order of expansions is: brace expansion, tilde expansion, parameter, variable and arithmetic expansion and command substitution (done in a left-to-right fashion), word splitting, and pathname expansion. On systems that can support it, there is an additional expansion available: process substitution. Furthermore, expansion absolutely does work on the first word and it's not obvious to me why someone might think it doesn't. The simplest example is probably tilde expansion, i.e. the ability execute ~/bin/foo or ~someone/bin/bar. But the other types work too, e.g. brace expansion: $ ls l1 ls: l1: No such file or directory $ l{s,1} ls: l1: No such file or directory $ touch l1 $ l{s,1} l1 Command substitution: $ $(echo l)s -l l1 -rw-r--r-- 1 user wheel 0 10 Dec 14:12 l1 Arithmetic expansion: $ perl$((4+1)).$((36/2)) -E 'say "foo"' foo Even pathname expansion - here, `fi*` expands to `find` from $CWD, which is then found in $PATH and executed: $ ls -l $ touch find foo bar quux $ fi* . -name f\* ./foo ./find And so on.*
- necovek 5y ago
  > ...expansion absolutely does work on the first word As was already pointed out, you are right. It does not work to match executables in the $PATH, which is what I was, for some reason, trying to communicate badly.
- necovek 5y ago
  > I think it's a mistake to refer to any of this as "glob expansion" Since we are being pedantic regarding the terms, while I can't really find "glob expansion" in the Bash manual either, it documents a bunch of "glob" settings: "noglob", "GLOBIGNORE", "dotglob", "extglob"...[1] FWIW, I expected to find these in the Bash info page on Ubuntu, but curiously, "info bash" does not show up the info page (just the fallback manpage) — "info find" does work. Whatever happened to (tex)info pages? [1]https://www.gnu.org/savannah-checkouts/gnu/bash/manual/bash.html#Pattern-Matching https://www.gnu.org/savannah-checkouts/gnu/bash/manual/bash....
- dredmorbius 5y ago
  The * character is addressed in pattern matching rather than parameter expansion. From the bash manpage (version 5.0): The special pattern characters have the following meanings: * Matches any string, including the null string. When the globstar shell option is enabled, and * is used in a pathname expansion context, two adjacent *s used as a single pattern will match all files and zero or more directories and subdirectories. If followed by a /, two adjacent *s will match only directories and subdirectories. The GNU bash manual describes the '-f' option as "Disable filename expansion (globbing)." "Globbing" is how * expansion is generally referenced in other documents, if not often within the bash manpage or info pages themselves.
GuB-42 5y ago
For me, it is a mess no matter how you look at it. It is because there are so many layers. The way commands are parsed is already not that easy, but then the command itself has its own rules, and it is not uncommon to pass a script as an argument. for example if you type "sh -c ls", first it is parsed into exec("sh", "-c", "ls"), then "sh" parses its arguments "-c" and "ls", then a sub-shell is started that parses the command "ls", then the actual "ls" program is launched. This is simple but at every step, there are some quirks, first is all the shell parsing issues, with globs, interpolation and all that. Then you have the issues of how the command parses its arguments (be careful about "-"). Then back to the shell parser with the sub-command, then "ls" itself has its quirks, like locales for instance. I have done Perl (line noise), C++ (memory unsafe with too many features), PHP (injections everywhere), JS (with IE6 support) and none of these languages make me more uncomfortable than UNIX shells.

formerly_proven 5y ago

You can actually use expansion even in the first word, just not for searching $PATH because that'd be inefficient.

necovek 5y ago
You are right, for some reason I was focused on the combined search-through-$PATH and glob expansion at the same time.

numerobis 5y ago

Are backticks necessary these days? I thought it was advised to use `$(...)` instead…

necovek 5y ago
If you are going to accidentally type them in (maybe pasting some markdown?), it's surely good to know what they do.

darrenf 5y ago

If we're talking about bash, I think it's a mistake to refer to any of this as "glob expansion", a term which doesn't appear in the bash man page. Rather, it differentiates between multiple types of expansion and the order in which they take place. Quote: The order of expansions is: brace expansion, tilde expansion, parameter, variable and arithmetic expansion and command substitution (done in a left-to-right fashion), word splitting, and pathname expansion. On systems that can support it, there is an additional expansion available: process substitution. Furthermore, expansion absolutely does work on the first word and it's not obvious to me why someone might think it doesn't. The simplest example is probably tilde expansion, i.e. the ability execute ~/bin/foo or ~someone/bin/bar. But the other types work too, e.g. brace expansion: $ ls l1 ls: l1: No such file or directory $ l{s,1} ls: l1: No such file or directory $ touch l1 $ l{s,1} l1 Command substitution: $ $(echo l)s -l l1 -rw-r--r-- 1 user wheel 0 10 Dec 14:12 l1 Arithmetic expansion: $ perl$((4+1)).$((36/2)) -E 'say "foo"' foo Even pathname expansion - here, `fi*` expands to `find` from $CWD, which is then found in $PATH and executed: $ ls -l $ touch find foo bar quux $ fi* . -name f\* ./foo ./find And so on.*

necovek 5y ago
> ...expansion absolutely does work on the first word As was already pointed out, you are right. It does not work to match executables in the $PATH, which is what I was, for some reason, trying to communicate badly.
necovek 5y ago
> I think it's a mistake to refer to any of this as "glob expansion" Since we are being pedantic regarding the terms, while I can't really find "glob expansion" in the Bash manual either, it documents a bunch of "glob" settings: "noglob", "GLOBIGNORE", "dotglob", "extglob"...[1] FWIW, I expected to find these in the Bash info page on Ubuntu, but curiously, "info bash" does not show up the info page (just the fallback manpage) — "info find" does work. Whatever happened to (tex)info pages? [1]https://www.gnu.org/savannah-checkouts/gnu/bash/manual/bash.html#Pattern-Matching https://www.gnu.org/savannah-checkouts/gnu/bash/manual/bash....
dredmorbius 5y ago
The * character is addressed in pattern matching rather than parameter expansion. From the bash manpage (version 5.0): The special pattern characters have the following meanings: * Matches any string, including the null string. When the globstar shell option is enabled, and * is used in a pathname expansion context, two adjacent *s used as a single pattern will match all files and zero or more directories and subdirectories. If followed by a /, two adjacent *s will match only directories and subdirectories. The GNU bash manual describes the '-f' option as "Disable filename expansion (globbing)." "Globbing" is how * expansion is generally referenced in other documents, if not often within the bash manpage or info pages themselves.

GuB-42 5y ago

For me, it is a mess no matter how you look at it. It is because there are so many layers. The way commands are parsed is already not that easy, but then the command itself has its own rules, and it is not uncommon to pass a script as an argument. for example if you type "sh -c ls", first it is parsed into exec("sh", "-c", "ls"), then "sh" parses its arguments "-c" and "ls", then a sub-shell is started that parses the command "ls", then the actual "ls" program is launched. This is simple but at every step, there are some quirks, first is all the shell parsing issues, with globs, interpolation and all that. Then you have the issues of how the command parses its arguments (be careful about "-"). Then back to the shell parser with the sub-command, then "ls" itself has its quirks, like locales for instance. I have done Perl (line noise), C++ (memory unsafe with too many features), PHP (injections everywhere), JS (with IE6 support) and none of these languages make me more uncomfortable than UNIX shells.