16 ms·

Understanding Bash (2018)

130 points by sergi_chalauri 7y ago

empath75 7y ago
> Note that the exit value of true is 0, and the exit value of false is 1. This is somewhat counterintuitive, and it's the exact opposite of most programming languages. "Happy families are all alike; every unhappy family is unhappy in its own way." https://en.wikipedia.org/wiki/Anna_Karenina_principle https://en.wikipedia.org/wiki/Anna_Karenina_principle
- AdmiralAsshat 7y ago
  It took me awhile to wrap my head around when I initially encountered it, but the idea made sense after I thought about it. If the process exited successfully, we don't care--it did what it was supposed to do. But if it failed, we'd probably like as much info about why it failed as possible; and so having the exit status be more like an error code clearly has some value.
  - elmigranto 7y ago
    Good point. I think the fact that it’s named “exit status” instead of “error code” is primary confusion factor.
  - empath75 7y ago
    I was being somewhat oblique about it but that was what my quote was saying.
- michaelcampbell 7y ago
  Some older versions of BASIC did this as well, except false was -1. I think internally it was represented as all 0 bits for true, all 1 bits for false. To be pendantic, at least in terms of bash if's, ['s, and test's, I think 0 is true and non-0 is non-true though, no?
- reportgunner 7y ago
  This is there to allow more than just true and false, like warnings and custom exit codes. https://stackoverflow.com/questions/4419952/difference-between-return-and-exit-in-bash-functions https://stackoverflow.com/questions/4419952/difference-betwe... I have also encountered this in C# https://docs.microsoft.com/en-us/dotnet/api/system.environment.exit?view=netframework-4.8 https://docs.microsoft.com/en-us/dotnet/api/system.environme...
- banachtarski 7y ago
  The article is wrong though. For error codes, 0 being a success value is the CONVENTION. A non-zero code is an error. This isn't about "programming languages."
  - gbacon 7y ago
    Wrong is a bit harsh for a convention or semantics, more like imprecisely worded.
  - SAI_Peregrinus 7y ago
    The problem is that for bash `true` is a function (or builtin) that returns 0, while for C (and thus most languages) any non-zero value (typically 1 for the builtins) is true. In c99 and later with stdbool.h, `true` is almost always defined as `#define true 1`.
  - aidenn0 7y ago
    the existence of the && and || operators (not to mention the true and false utilities) would indicate it's more than just a convention.
- flingo 7y ago
  I certainly prefer this to the reverse, and having truth values and return values mean something different.
fooblat 7y ago
I love bash and I know I'm not alone. Sure, it's syntax is based on ideas from the 1960s[0] and it has a weird core library of functions[1]. Certainly writing code that is a blend of logic and managing other programs takes some getting used to. However, it is very well documented[2] and I've had great success with it over the past 20 years. If one takes the time to get to know it, it is actually easy and fun to write scripts that are robust and easy to maintain. I'd venture to say that bash is the putty in the little gaps of the internet. It is the fitting that glues programs to the various systems on which they run. 0. https://en.wikipedia.org/wiki/ALGOL https://en.wikipedia.org/wiki/ALGOL 1. https://en.wikipedia.org/wiki/POSIX https://en.wikipedia.org/wiki/POSIX 2. https://www.gnu.org/software/bash/manual/ https://www.gnu.org/software/bash/manual/ edit: formatting
- bordercases 7y ago
  It's also extremely fast.
  - kazinator 7y ago
    Sure, until you need to do something advanced like have a function return a string and use it in the caller. Test case: #!/bin/bash fun() { echo "result" } RES=$(fun) echo $RES Snippet from system call trace (via "strace bash ./test.sh"): pipe([3, 4]) = 0 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 rt_sigprocmask(SIG_BLOCK, [INT CHLD], [], 8) = 0 lseek(255, -10, SEEK_CUR) = 51 clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7fccbed72a10) = 3870 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 rt_sigaction(SIGCHLD, {0x446240, [], SA_RESTORER|SA_RESTART, 0x7fccbe3b0ff0}, {0x446240, [], SA_RESTORER|SA_RESTART, 0x7fccbe3b0ff0}, 8) = 0 close(4) = 0 read(3, "result\n", 128) = 7 read(3, "", 128) = 0 In other words, this requires a pipe to be created a child process to be spawned, and its output to be read from that pipe. In any reasonable scripting language, and even some unreasonable ones, this is all done in one address space. How about, oh, Awk: function fun() { return "result" } One of the most common mantras for effective shell programming is "avoid writing loops; orchestrate external utilities".
m4r35n357 7y ago
It is far simpler to just learn the POSIX shell first - much shorter man page. The the Bash stuff is then just a few additions if you ever need them (other shells are available).
- frou_dh 7y ago
  Notably, such a concept as a function having a local variable is beyond POSIX shell.
  - m4r35n357 7y ago
    "Variables may be declared to be local to a function by using a 'local' command."
    - frou_dh 7y ago
      Here's the spec. Where is it? https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V...
      - m4r35n357 7y ago
        It is in both ash and dash, the shells I mentioned. Your attempt at point scoring is irrelevant but expected. ash & dash are about 100kB executables (POSIX plus a tiny number of non-interactive improvments like 'local'), bash is 1100kB. Every shell and subshell.
        frou_dh 7y ago
        My point is that POSIX shell purity is masochism. Understanding how it's the core and other shells build on top of it is certainly valuable, but it's not inherently particularly good to use.
        m4r35n357 7y ago
JoelMcCracken 7y ago
This seems like a very basic introduction to bash, which is fine, but I was hoping for something called "Understanding Bash" to help with some of the understanding about why it behaves so weirdly at times. For me, the biggest problem when writing bash is when I need to do something just a little more complex than what is there, but there is no good way to do it. An example that usually trips me up is this: https://mywiki.wooledge.org/BashFAQ/050 https://mywiki.wooledge.org/BashFAQ/050. FWIW, the above link is probably the most useful thing I have found that actually helps me understand bash.
- gbacon 7y ago
  “Too much for bash, Goose; I’m switching to Perl!”
- nixpulvis 7y ago
  I've been writing a (hopefully one day) POSIX compatible shell using a parser generator library. Having to try and fit sh into a parser framework makes you make some choices, and it's a great way to find these weird things. For example, I was surprised at first to learn that `{ ls }` isn't a valid program. Or: FOO=1 echo $FOO # vs FOO=1; echo $FOO # vs FOO=1 printenv FOO
- fanf2 7y ago
  If you think you might want to use arrays in bash, or any kind of structured data, stop. If your script is getting much more than 100 lines, stop. Rewrite it in a better programming language before it is too late!
ReedJessen 7y ago
This is a really good article and worth the read.
peterwwillis 7y ago
> [ ] is a command—basically another way to call the built-in test command. Literally, '/usr/bin/[', which takes as an argument a final ']', but actually a different binary than '/usr/bin/test'. And is different from the bash builtin. Crazy shit. vagrant@vagrant:~$ [ --version -bash: [: missing `]' vagrant@vagrant:~$ /usr/bin/[ --version [ (GNU coreutils) 8.28 Copyright (C) 2017 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>. This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Written by Kevin Braunsdorf and Matthew Bradburn.
- kohtatsu 7y ago
  I think it's cute.
peterwwillis 7y ago
I've been using bash for nearly two decades, and I just found out that Bash is apparently the only shell whose built-in echo uses the -e option. I thought others used this too, but it seems every other shell's echo just implicitly interpolates escaped characters. POSIX actually says anything following '\' is undefined behavior. Apparently printf is the only portable way to interpolate escaped characters in output. Here's a bunch of very useful tips like that: https://www.etalabs.net/sh_tricks.html https://www.etalabs.net/sh_tricks.html
- kohtatsu 7y ago
  That resource is lovely, thank you.
lind 7y ago
Very interesting read, thanks
RMPR 7y ago
Amazing article, I like to understand the why behind things like that. Kudos to you.
RMPR 7y ago
> Second, in principle, there's nothing to enforce that a UNIX shell must have echo as a built-in, and therefore, it's important to have the external utility /bin/echo as a fallback. I read here https://en.wikipedia.org/wiki/POSIX#Overview https://en.wikipedia.org/wiki/POSIX#Overview that echo has been standardized for POSIX, can you elaborate further ?

> Note that the exit value of true is 0, and the exit value of false is 1. This is somewhat counterintuitive, and it's the exact opposite of most programming languages. "Happy families are all alike; every unhappy family is unhappy in its own way." https://en.wikipedia.org/wiki/Anna_Karenina_principle https://en.wikipedia.org/wiki/Anna_Karenina_principle

AdmiralAsshat 7y ago
It took me awhile to wrap my head around when I initially encountered it, but the idea made sense after I thought about it. If the process exited successfully, we don't care--it did what it was supposed to do. But if it failed, we'd probably like as much info about why it failed as possible; and so having the exit status be more like an error code clearly has some value.
- elmigranto 7y ago
  Good point. I think the fact that it’s named “exit status” instead of “error code” is primary confusion factor.
- empath75 7y ago
  I was being somewhat oblique about it but that was what my quote was saying.
michaelcampbell 7y ago
Some older versions of BASIC did this as well, except false was -1. I think internally it was represented as all 0 bits for true, all 1 bits for false. To be pendantic, at least in terms of bash if's, ['s, and test's, I think 0 is true and non-0 is non-true though, no?
reportgunner 7y ago
This is there to allow more than just true and false, like warnings and custom exit codes. https://stackoverflow.com/questions/4419952/difference-between-return-and-exit-in-bash-functions https://stackoverflow.com/questions/4419952/difference-betwe... I have also encountered this in C# https://docs.microsoft.com/en-us/dotnet/api/system.environment.exit?view=netframework-4.8 https://docs.microsoft.com/en-us/dotnet/api/system.environme...
banachtarski 7y ago
The article is wrong though. For error codes, 0 being a success value is the CONVENTION. A non-zero code is an error. This isn't about "programming languages."
- gbacon 7y ago
  Wrong is a bit harsh for a convention or semantics, more like imprecisely worded.
- SAI_Peregrinus 7y ago
  The problem is that for bash `true` is a function (or builtin) that returns 0, while for C (and thus most languages) any non-zero value (typically 1 for the builtins) is true. In c99 and later with stdbool.h, `true` is almost always defined as `#define true 1`.
- aidenn0 7y ago
  the existence of the && and || operators (not to mention the true and false utilities) would indicate it's more than just a convention.
flingo 7y ago
I certainly prefer this to the reverse, and having truth values and return values mean something different.

fooblat 7y ago

I love bash and I know I'm not alone. Sure, it's syntax is based on ideas from the 1960s[0] and it has a weird core library of functions[1]. Certainly writing code that is a blend of logic and managing other programs takes some getting used to. However, it is very well documented[2] and I've had great success with it over the past 20 years. If one takes the time to get to know it, it is actually easy and fun to write scripts that are robust and easy to maintain. I'd venture to say that bash is the putty in the little gaps of the internet. It is the fitting that glues programs to the various systems on which they run. 0. https://en.wikipedia.org/wiki/ALGOL https://en.wikipedia.org/wiki/ALGOL 1. https://en.wikipedia.org/wiki/POSIX https://en.wikipedia.org/wiki/POSIX 2. https://www.gnu.org/software/bash/manual/ https://www.gnu.org/software/bash/manual/ edit: formatting

bordercases 7y ago
It's also extremely fast.
- kazinator 7y ago
  Sure, until you need to do something advanced like have a function return a string and use it in the caller. Test case: #!/bin/bash fun() { echo "result" } RES=$(fun) echo $RES Snippet from system call trace (via "strace bash ./test.sh"): pipe([3, 4]) = 0 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 rt_sigprocmask(SIG_BLOCK, [INT CHLD], [], 8) = 0 lseek(255, -10, SEEK_CUR) = 51 clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7fccbed72a10) = 3870 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 rt_sigaction(SIGCHLD, {0x446240, [], SA_RESTORER|SA_RESTART, 0x7fccbe3b0ff0}, {0x446240, [], SA_RESTORER|SA_RESTART, 0x7fccbe3b0ff0}, 8) = 0 close(4) = 0 read(3, "result\n", 128) = 7 read(3, "", 128) = 0 In other words, this requires a pipe to be created a child process to be spawned, and its output to be read from that pipe. In any reasonable scripting language, and even some unreasonable ones, this is all done in one address space. How about, oh, Awk: function fun() { return "result" } One of the most common mantras for effective shell programming is "avoid writing loops; orchestrate external utilities".

m4r35n357 7y ago

It is far simpler to just learn the POSIX shell first - much shorter man page. The the Bash stuff is then just a few additions if you ever need them (other shells are available).

frou_dh 7y ago
Notably, such a concept as a function having a local variable is beyond POSIX shell.
- m4r35n357 7y ago
  "Variables may be declared to be local to a function by using a 'local' command."
  - frou_dh 7y ago
    Here's the spec. Where is it? https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V...
    - m4r35n357 7y ago
      It is in both ash and dash, the shells I mentioned. Your attempt at point scoring is irrelevant but expected. ash & dash are about 100kB executables (POSIX plus a tiny number of non-interactive improvments like 'local'), bash is 1100kB. Every shell and subshell.
      - frou_dh 7y ago
        My point is that POSIX shell purity is masochism. Understanding how it's the core and other shells build on top of it is certainly valuable, but it's not inherently particularly good to use.
        m4r35n357 7y ago

JoelMcCracken 7y ago

This seems like a very basic introduction to bash, which is fine, but I was hoping for something called "Understanding Bash" to help with some of the understanding about why it behaves so weirdly at times. For me, the biggest problem when writing bash is when I need to do something just a little more complex than what is there, but there is no good way to do it. An example that usually trips me up is this: https://mywiki.wooledge.org/BashFAQ/050 https://mywiki.wooledge.org/BashFAQ/050. FWIW, the above link is probably the most useful thing I have found that actually helps me understand bash.

gbacon 7y ago
“Too much for bash, Goose; I’m switching to Perl!”
nixpulvis 7y ago
I've been writing a (hopefully one day) POSIX compatible shell using a parser generator library. Having to try and fit sh into a parser framework makes you make some choices, and it's a great way to find these weird things. For example, I was surprised at first to learn that `{ ls }` isn't a valid program. Or: FOO=1 echo $FOO # vs FOO=1; echo $FOO # vs FOO=1 printenv FOO
fanf2 7y ago
If you think you might want to use arrays in bash, or any kind of structured data, stop. If your script is getting much more than 100 lines, stop. Rewrite it in a better programming language before it is too late!

ReedJessen 7y ago

This is a really good article and worth the read.

peterwwillis 7y ago

> [ ] is a command—basically another way to call the built-in test command. Literally, '/usr/bin/[', which takes as an argument a final ']', but actually a different binary than '/usr/bin/test'. And is different from the bash builtin. Crazy shit. vagrant@vagrant:~$ [ --version -bash: [: missing `]' vagrant@vagrant:~$ /usr/bin/[ --version [ (GNU coreutils) 8.28 Copyright (C) 2017 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>. This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Written by Kevin Braunsdorf and Matthew Bradburn.

kohtatsu 7y ago
I think it's cute.

peterwwillis 7y ago

I've been using bash for nearly two decades, and I just found out that Bash is apparently the only shell whose built-in echo uses the -e option. I thought others used this too, but it seems every other shell's echo just implicitly interpolates escaped characters. POSIX actually says anything following '\' is undefined behavior. Apparently printf is the only portable way to interpolate escaped characters in output. Here's a bunch of very useful tips like that: https://www.etalabs.net/sh_tricks.html https://www.etalabs.net/sh_tricks.html

kohtatsu 7y ago
That resource is lovely, thank you.

lind 7y ago

Very interesting read, thanks

RMPR 7y ago

Amazing article, I like to understand the why behind things like that. Kudos to you.

RMPR 7y ago

> Second, in principle, there's nothing to enforce that a UNIX shell must have echo as a built-in, and therefore, it's important to have the external utility /bin/echo as a fallback. I read here https://en.wikipedia.org/wiki/POSIX#Overview https://en.wikipedia.org/wiki/POSIX#Overview that echo has been standardized for POSIX, can you elaborate further ?