4 ms·

I'm working on fixing errexit / set -e in Oil: https://github.com/oilshell/oil/issues/709 https://github.com/oilshell/oil/issues/709 Summary of problems: 1.

by chubot 6y ago

I'm working on fixing errexit / set -e in Oil: https://github.com/oilshell/oil/issues/709 https://github.com/oilshell/oil/issues/709 Summary of problems: 1. The "if myfunc" problem -- error checking is skipped. This is specified by POSIX, but it's undesirable. 2. The f() { test -d /tmp && echo "exists" } problem. The exit code of the function is probably not what you expect! 3. the local x=$(false) problem. This happens in all shells because of the definition of $?. (local is a shell builtin with its own exit code, not part of the language) This one is already fixed with shopt -s more_errexit in Oil. 4. the x=$(false) problem. This is a bash problem with command substitution. For example, dash and zsh don't have this problem. Test case: bash -c 'set -e; x=$(false); echo should not get here' Newer versions of bash fix it with inherit_errexit. Oil also implements inherit_errexit and turns it on by default! (in Oil, not OSH) ----- So 1 and 2 relate to the confusing combination of shell functions and errexit. And 3 and 4 relate to command subs. Oil has fixed these with opt-in OSH shell options (on by default in Oil), but not 1 and 2. If you know of any other problems, please chime in on the bug!

a1369209993 6y ago
> 2. The f() { test -d /tmp && echo "exists" } problem. The exit code of the function is probably not what you expect! The exit code is 0 (assuming /tmp/, stdout, /bin/test and /bin/echo are all working correctly; with /tmp1 it's 1), as expected; is this referencing a bug in sh and/or bash that I've fixed locally and then forgotten about? (Also, I'm pretty sure it should be: f() { test -d /tmp && echo "exists"; } unless the parse error for missing ';' was your point (I haven't bothered to fix that one, but maybe Oil has).)
- chubot 6y ago
  Yeah I didn't explain that very well. It's actually a variation on the problem here: https://news.ycombinator.com/item?id=24738274 https://news.ycombinator.com/item?id=24738274 except with a shell function rather than a subshell. Demo: https://github.com/oilshell/blog-code/blob/master/errexit https://github.com/oilshell/blog-code/blob/master/errexit /demo.sh#L21 Run ./demo.sh subshell-compare and func-compare. I have seen multiple people point it out as confusing. The problem is that in the first case, there are two exit codes: 1. false (suppressed on LHS of &&) 2. true And in the second case, there are 3: 1. false (suppressed on LHS of &&) 2. true 3. the exit code of the function/subshell, which is the exit code of the last statement, which is nonzero. This then CAUSES THE PROGRAM TO FAIL, whereas it didn't before. (edit: removed incorrect code) ----- So it's kind of like a "ghost" exit code created by the subshell/function! Which has an unexpected interaction with errexit. Does that make sense? I am not sure exactly how to fix it, but it will probably involve limiting && to stuff like this: if test -d /tmp && test -d /tmp/foo; then echo yes; fi And disallow it when standing alone, because it doesn't make much sense there, if errexit is on. Feel free to chime in bug with any possible solutions or more problems. ---- (And yes Oil fixes the brace problem with shopt -s parse_brace, which is on by default in Oil! Try it out and let me know if you like it :) )
  - a1369209993 6y ago
    > Yeah I didn't explain that very well. Not at all well, I'm afraid, but I think I understand: f(){ false && true; } ; set -e ; f ; echo hi # this works correctly (f returns false and errexit triggers) set -e; false && true ; echo hi # despite `false && true` failing, this doesn't trigger errexit I was confused by the implication that the function version was what was wrong (rather than "false && true is broken, and you'd expect the function to work the same way"). Probably partly because I was confusing it with `command || true`, which is a idiomatic way to suppress errexit-like mechanisms by making the exit status always 0.
    - chubot 6y ago
      Well, which one is broken depends on your viewpoint, and the exact use case. Another way to look at it is the difference between exercise 4 and 5 here: https://mywiki.wooledge.org/BashFAQ/105 https://mywiki.wooledge.org/BashFAQ/105 So taking that concrete example test -d nosuchdir && echo "warning: no directory"; instead of false && true You didn't want the function to fail. You just wanted to print a warning if the directory doesn't exist. But the whole function fails, while it doesn't for the if statement. The whole thing is inherently confusing ... The language confuses success/failure and true/false. Both of those are stuffed into an exit code. test -d is really for true/false, but then functions also have success/failure (did the last command succeed). So there's no way for shell to really know what you want. I think Oil might end up with something like "if catch foo" for success/fail, and "if boolean grep" for true/false.
      - a1369209993 6y ago
        > You just wanted to print a warning if the directory doesn't exist. That code prints a warning if the directory does exist, actually ("-d FILE FILE exists and is a directory"). Did you mean something like: test -e outfile && echo "trying to overwrite preexisting outfile" > You didn't want the function to fail. Uh, I kind of do, actually? If I wanted to discard the exit status I'd write: test -e outfile && echo "trying to overwrite preexisting outfile" || true; This is awkward, and suggests a proper if-then operator (perl-6-style `??` collides with globbing, but maybe `&?`?), but it's better then having something like: mkfoo -o foo && cp -T foo otherfoo do-something foo bar mostly-silently mix old and new data together because mkfoo failed and the error code was swallowed. > So there's no way for shell to really know what you want. Yep, and if we have to pick one problem, things failing completely in a obvious and also easily-fixable way is usually a much less awful problem to have than silently corrupting data and state, especially when you've opted-in to a mechanism (like errexit) specifically designed to do the former.

by chubot 6y ago

a1369209993 6y ago

> 2. The f() { test -d /tmp && echo "exists" } problem. The exit code of the function is probably not what you expect! The exit code is 0 (assuming /tmp/, stdout, /bin/test and /bin/echo are all working correctly; with /tmp1 it's 1), as expected; is this referencing a bug in sh and/or bash that I've fixed locally and then forgotten about? (Also, I'm pretty sure it should be: f() { test -d /tmp && echo "exists"; } unless the parse error for missing ';' was your point (I haven't bothered to fix that one, but maybe Oil has).)

chubot 6y ago
Yeah I didn't explain that very well. It's actually a variation on the problem here: https://news.ycombinator.com/item?id=24738274 https://news.ycombinator.com/item?id=24738274 except with a shell function rather than a subshell. Demo: https://github.com/oilshell/blog-code/blob/master/errexit https://github.com/oilshell/blog-code/blob/master/errexit /demo.sh#L21 Run ./demo.sh subshell-compare and func-compare. I have seen multiple people point it out as confusing. The problem is that in the first case, there are two exit codes: 1. false (suppressed on LHS of &&) 2. true And in the second case, there are 3: 1. false (suppressed on LHS of &&) 2. true 3. the exit code of the function/subshell, which is the exit code of the last statement, which is nonzero. This then CAUSES THE PROGRAM TO FAIL, whereas it didn't before. (edit: removed incorrect code) ----- So it's kind of like a "ghost" exit code created by the subshell/function! Which has an unexpected interaction with errexit. Does that make sense? I am not sure exactly how to fix it, but it will probably involve limiting && to stuff like this: if test -d /tmp && test -d /tmp/foo; then echo yes; fi And disallow it when standing alone, because it doesn't make much sense there, if errexit is on. Feel free to chime in bug with any possible solutions or more problems. ---- (And yes Oil fixes the brace problem with shopt -s parse_brace, which is on by default in Oil! Try it out and let me know if you like it :) )
- a1369209993 6y ago
  > Yeah I didn't explain that very well. Not at all well, I'm afraid, but I think I understand: f(){ false && true; } ; set -e ; f ; echo hi # this works correctly (f returns false and errexit triggers) set -e; false && true ; echo hi # despite `false && true` failing, this doesn't trigger errexit I was confused by the implication that the function version was what was wrong (rather than "false && true is broken, and you'd expect the function to work the same way"). Probably partly because I was confusing it with `command || true`, which is a idiomatic way to suppress errexit-like mechanisms by making the exit status always 0.
  - chubot 6y ago
    Well, which one is broken depends on your viewpoint, and the exact use case. Another way to look at it is the difference between exercise 4 and 5 here: https://mywiki.wooledge.org/BashFAQ/105 https://mywiki.wooledge.org/BashFAQ/105 So taking that concrete example test -d nosuchdir && echo "warning: no directory"; instead of false && true You didn't want the function to fail. You just wanted to print a warning if the directory doesn't exist. But the whole function fails, while it doesn't for the if statement. The whole thing is inherently confusing ... The language confuses success/failure and true/false. Both of those are stuffed into an exit code. test -d is really for true/false, but then functions also have success/failure (did the last command succeed). So there's no way for shell to really know what you want. I think Oil might end up with something like "if catch foo" for success/fail, and "if boolean grep" for true/false.
    - a1369209993 6y ago
      > You just wanted to print a warning if the directory doesn't exist. That code prints a warning if the directory does exist, actually ("-d FILE FILE exists and is a directory"). Did you mean something like: test -e outfile && echo "trying to overwrite preexisting outfile" > You didn't want the function to fail. Uh, I kind of do, actually? If I wanted to discard the exit status I'd write: test -e outfile && echo "trying to overwrite preexisting outfile" || true; This is awkward, and suggests a proper if-then operator (perl-6-style `??` collides with globbing, but maybe `&?`?), but it's better then having something like: mkfoo -o foo && cp -T foo otherfoo do-something foo bar mostly-silently mix old and new data together because mkfoo failed and the error code was swallowed. > So there's no way for shell to really know what you want. Yep, and if we have to pick one problem, things failing completely in a obvious and also easily-fixable way is usually a much less awful problem to have than silently corrupting data and state, especially when you've opted-in to a mechanism (like errexit) specifically designed to do the former.