3 ms·

> Running things in parallel is literally a single `&` character. Yes now try waiting for all parallel tasks, and error out if at least one errors. And try sep

> Running things in parallel is literally a single `&` character. Yes now try waiting for all parallel tasks, and error out if at least one errors. And try separating their output so that they don't get interleaved into a big mess. And try by default hiding the outputs of commands that succeeded, only showing those that failed, except when the user explicitly asks for it. Your "simple" shell script now suddenly isn't so simple anymore.

stabbles 4y ago
Using GNU make you can get very nice output sync: all: variant-1 variant-2 @echo done with all variant-%: @echo starting $@ @sleep 1 @echo done $@ Use `make -j2 --output-sync=target`.
- notpushkin 4y ago
  brew install make gmake -j2 --output-sync=target on macOS.
cormacrelf 4y ago
That’s what GNU Parallel is for. Or Pueue, which gives you a GH-Actions-level feature set but it’s less likely to be installed on any particular machine. Pretty sure you could fetch a pueue binary at the start of an actions script and do everything that way. (These can’t project their tasks over multiple GH Actions runners, eg for multiple OSes. For that you will need to use the YAML. Good compilers will already do work in parallel and max out however many cores they are given, so multi machine is the main use case. Unfortunate.)
Arnavion 4y ago
>Yes now try waiting for all parallel tasks, `wait` >and error out if at least one errors. `wait` propagates the exit status of the waited task. `set -e` triggers the script to exit on any command failure. >And try separating their output so that they don't get interleaved into a big mess. `| while read -r line; do echo "task X: $line"; done` Learn the tools instead of being so confident that it can't be done or that it's complicated.
- FooBarWidget 4y ago
  You remind me of that guy who said there's no point in Dropbox because one can "just simply setup an sftp server".

stabbles 4y ago

Using GNU make you can get very nice output sync: all: variant-1 variant-2 @echo done with all variant-%: @echo starting $@ @sleep 1 @echo done $@ Use `make -j2 --output-sync=target`.

notpushkin 4y ago
brew install make gmake -j2 --output-sync=target on macOS.

cormacrelf 4y ago

That’s what GNU Parallel is for. Or Pueue, which gives you a GH-Actions-level feature set but it’s less likely to be installed on any particular machine. Pretty sure you could fetch a pueue binary at the start of an actions script and do everything that way. (These can’t project their tasks over multiple GH Actions runners, eg for multiple OSes. For that you will need to use the YAML. Good compilers will already do work in parallel and max out however many cores they are given, so multi machine is the main use case. Unfortunate.)

Arnavion 4y ago

>Yes now try waiting for all parallel tasks, `wait` >and error out if at least one errors. `wait` propagates the exit status of the waited task. `set -e` triggers the script to exit on any command failure. >And try separating their output so that they don't get interleaved into a big mess. `| while read -r line; do echo "task X: $line"; done` Learn the tools instead of being so confident that it can't be done or that it's complicated.

FooBarWidget 4y ago
You remind me of that guy who said there's no point in Dropbox because one can "just simply setup an sftp server".