5 ms·

Some additional tips: 1. You don't need the parentheses. 2. If you use process substitution [1] instead of a pipe, you will stay in the same process and can m

Some additional tips: 1. You don't need the parentheses. 2. If you use process substitution [1] instead of a pipe, you will stay in the same process and can modify variables of the enclosing scope: i=0 while read -r v; do ... i=$(( i + 1)) done < <(do_something) The drawback is that this way `do_something` has to come after `done`, but that's bash for you ¯\_(ツ)_/¯ [1] https://www.gnu.org/software/bash/manual/html_node/Process-Substitution.html https://www.gnu.org/software/bash/manual/html_node/Process-S...

fiddlerwoaroof 5y ago
Yeah, although I use the parentheses mostly because I like how it reads. And that process substitution trick is important too. I think the redirection can come first, though (not at a computer to test): < <( do_something ) while read . . .
- aaaaaaaaaaab 5y ago
  Yeah, for commands, the input/output redirections can precede them, but for some reason it doesn't work for builtin constructs like `while`: $ < <( echo foo ) while read -r f; do echo "$f"; done -bash: syntax error near unexpected token `do' $ < <( echo foo ) xargs echo foo $ bash --version GNU bash, version 5.1.4(1)-release (x86_64-apple-darwin20.2.0)
  - fiddlerwoaroof 5y ago
    Maybe wrap the loop either with parentheses or braces?
    - aaaaaaaaaaab 5y ago
      Tried that, but nope :D I'll let you figure this one out once you get near a computer!
- entire-name 5y ago
  Redirection like this doesn't seem to work if it comes first on GNU bash 5.0.17(1)-release. For documentation purposes, this is the exact thing I tried to run: $ < <(echo hi) while read a; do echo "got $a"; done -bash: syntax error near unexpected token `do' $ while read a; do echo "got $a"; done < <(echo hi) got hi Maybe there is another way...
  - JNRowe 5y ago
    One way which isn't great, but an option nonetheless… The zsh parser is happy with that form: $ zsh -c '< <(echo hi) while read a; do echo "got $a"; done' got hi My position isn't that it is a good reason to switch shells, but if you're using it anyway then it is an option.
    - fiddlerwoaroof 5y ago
      I’ve always preferred zsh and, as I’ve slowly adopted nix, I’ve slowly stopped writing bash in favor of zsh
- deleted 5y ago
  [deleted]
- lottin 5y ago
  This is not POSIX compliant though.
  - fiddlerwoaroof 5y ago
    These days bash and/or zsh are available nearly every place I care about, so I find POSIX compliance to be much less relevant.
    - pgtan 5y ago
      No, process substitution must be provided by the kernel/syslibs, it is not feature of bash. For example there is bash on AIX, but process substitution is not possible because the OS do not support it.
      - nonesuchluck 5y ago
        ksh93 depends exclusively on the kernel implementation of /dev/fd devices. I just checked `cat <(ls)` a moment ago on both Linux and AIX 7.2--the latter fails in ksh93t+. Bash uses /dev/fd when available, but also appears to have an internal implementation which silently creates named pipes and cleans them up. In Bash 5.0.18 on AIX, fake process substitution works just fine, in my testing.
        pgtan 5y ago
        Yes, you are right. Bash 5 on AIX 7.2 works with process substitution. Thanks for the advise!
chriswarbo 5y ago
I use this exact pattern a lot. One thing to consider is that in the process substitution version, do_something can't modify the enclosing variables. The vast majority of the time I want to modify variables in the loop body and not the generating process, but it's worth keeping in mind. One common pattern I use this for is running a bunch of checks/tests, e.g. EXIT_CODE=0 while read -r F do do_check "$F" || EXIT_CODE=1 done < <(find ./tests -type f) exit "$EXIT_CODE" This is a more complicated alternative to the following: find ./tests -type f | while read -r F do do_check "$F" || exit 1 done The simpler version will abort on the first error, whilst the first version will always run all of the checks (exiting with an error afterwards, if any of them failed)
- fiddlerwoaroof 5y ago
  I usually write zsh scripts and I think there’s a shell option in zsh that allows the loop at the end of the pipe to modify variables in the enclosing body: I remember at least one occasion where I was surprised about this discrepancy between shells.
  - aaaaaaaaaaab 5y ago
    Interesting! Indeed, Greg's BashFAQ notes it too: https://mywiki.wooledge.org/BashFAQ/024 https://mywiki.wooledge.org/BashFAQ/024 >Different shells exhibit different behaviors in this situation: >- BourneShell creates a subshell when the input or output of anything (loops, case etc..) but a simple command is redirected, either by using a pipeline or by a redirection operator ('<', '>'). >- BASH, Yash and PDKsh-derived shells create a new process only if the loop is part of a pipeline. >- KornShell and Zsh creates it only if the loop is part of a pipeline, but not if the loop is the last part of it. The read example above actually works in ksh88, ksh93, zsh! (but not MKsh or other PDKsh-derived shells) >- POSIX specifies the bash behaviour, but as an extension allows any or all of the parts of the pipeline to run without a subshell (thus permitting the KornShell behaviour, as well).
    - vbezhenar 5y ago

fiddlerwoaroof 5y ago

Yeah, although I use the parentheses mostly because I like how it reads. And that process substitution trick is important too. I think the redirection can come first, though (not at a computer to test): < <( do_something ) while read . . .

aaaaaaaaaaab 5y ago
Yeah, for commands, the input/output redirections can precede them, but for some reason it doesn't work for builtin constructs like `while`: $ < <( echo foo ) while read -r f; do echo "$f"; done -bash: syntax error near unexpected token `do' $ < <( echo foo ) xargs echo foo $ bash --version GNU bash, version 5.1.4(1)-release (x86_64-apple-darwin20.2.0)
- fiddlerwoaroof 5y ago
  Maybe wrap the loop either with parentheses or braces?
  - aaaaaaaaaaab 5y ago
    Tried that, but nope :D I'll let you figure this one out once you get near a computer!
entire-name 5y ago
Redirection like this doesn't seem to work if it comes first on GNU bash 5.0.17(1)-release. For documentation purposes, this is the exact thing I tried to run: $ < <(echo hi) while read a; do echo "got $a"; done -bash: syntax error near unexpected token `do' $ while read a; do echo "got $a"; done < <(echo hi) got hi Maybe there is another way...
- JNRowe 5y ago
  One way which isn't great, but an option nonetheless… The zsh parser is happy with that form: $ zsh -c '< <(echo hi) while read a; do echo "got $a"; done' got hi My position isn't that it is a good reason to switch shells, but if you're using it anyway then it is an option.
  - fiddlerwoaroof 5y ago
    I’ve always preferred zsh and, as I’ve slowly adopted nix, I’ve slowly stopped writing bash in favor of zsh
deleted 5y ago
[deleted]
lottin 5y ago
This is not POSIX compliant though.
- fiddlerwoaroof 5y ago
  These days bash and/or zsh are available nearly every place I care about, so I find POSIX compliance to be much less relevant.
  - pgtan 5y ago
    No, process substitution must be provided by the kernel/syslibs, it is not feature of bash. For example there is bash on AIX, but process substitution is not possible because the OS do not support it.
    - nonesuchluck 5y ago
      ksh93 depends exclusively on the kernel implementation of /dev/fd devices. I just checked `cat <(ls)` a moment ago on both Linux and AIX 7.2--the latter fails in ksh93t+. Bash uses /dev/fd when available, but also appears to have an internal implementation which silently creates named pipes and cleans them up. In Bash 5.0.18 on AIX, fake process substitution works just fine, in my testing.
      - pgtan 5y ago
        Yes, you are right. Bash 5 on AIX 7.2 works with process substitution. Thanks for the advise!

chriswarbo 5y ago

I use this exact pattern a lot. One thing to consider is that in the process substitution version, do_something can't modify the enclosing variables. The vast majority of the time I want to modify variables in the loop body and not the generating process, but it's worth keeping in mind. One common pattern I use this for is running a bunch of checks/tests, e.g. EXIT_CODE=0 while read -r F do do_check "$F" || EXIT_CODE=1 done < <(find ./tests -type f) exit "$EXIT_CODE" This is a more complicated alternative to the following: find ./tests -type f | while read -r F do do_check "$F" || exit 1 done The simpler version will abort on the first error, whilst the first version will always run all of the checks (exiting with an error afterwards, if any of them failed)

fiddlerwoaroof 5y ago
I usually write zsh scripts and I think there’s a shell option in zsh that allows the loop at the end of the pipe to modify variables in the enclosing body: I remember at least one occasion where I was surprised about this discrepancy between shells.
- aaaaaaaaaaab 5y ago
  Interesting! Indeed, Greg's BashFAQ notes it too: https://mywiki.wooledge.org/BashFAQ/024 https://mywiki.wooledge.org/BashFAQ/024 >Different shells exhibit different behaviors in this situation: >- BourneShell creates a subshell when the input or output of anything (loops, case etc..) but a simple command is redirected, either by using a pipeline or by a redirection operator ('<', '>'). >- BASH, Yash and PDKsh-derived shells create a new process only if the loop is part of a pipeline. >- KornShell and Zsh creates it only if the loop is part of a pipeline, but not if the loop is the last part of it. The read example above actually works in ksh88, ksh93, zsh! (but not MKsh or other PDKsh-derived shells) >- POSIX specifies the bash behaviour, but as an extension allows any or all of the parts of the pipeline to run without a subshell (thus permitting the KornShell behaviour, as well).
  - vbezhenar 5y ago