3 ms·
You can definitely do parallel / concurrent / async programming directly in the shell, without any helper programs. It's not exactly ergonomic, but with a few p
by fwip 7y ago
You can definitely do parallel / concurrent / async programming directly in the shell, without any helper programs. It's not exactly ergonomic, but with a few patterns it's not too difficult.
Async:
long_running_cmd & > results.txt
cmd_pid=$!
# other stuff
wait $cmd_pid
# do something with results, if you want
Parallel:
for script in "${scripts[@]}" ; do
"$script" &
done
wait
Dataflow:
mkfifo my_channel
mkfifo another_channel
my_cmd > my_channel &
tee my_channel \
>(cmd1) \
>(cmd2 > another_channel) &
another_cmd < another_channel
But if you want a useful dataflow environment, I really recommend checking out [Nextflow](https://www.nextflow.io/ https://www.nextflow.io/). I use it at work for constructing bioinformatics pipelines, and it's really natural. The "preview" DSL2 is worth looking at, as well.
- dmos62 7y agoThanks you for the recommendation, Nextflow looks exactly like what I was looking for! Thanks for the bash examples as well.