2 ms·
Ah, simply saying "parallelize" put me on the wrong track. I didn't realize you wanted to limit the number running simultaneously. Cool you discovered that xar
by RiderOfGiraffes 17y ago
Ah, simply saying "parallelize" put me on the wrong track. I didn't realize you wanted to limit the number running simultaneously. Cool you discovered that xargs can do it - I didn't know that.
I have, however, done this before. I have a "load_wait" command which waits for machine load to drop below a certain level before launching the command. A simple "sleep 1" between instances of that then lets me launch thousands of commands without killing the machine.
sleep 1 ; load_wait 3 <command1> &
sleep 1 ; load_wait 3 <command2> &
sleep 1 ; load_wait 3 <command3> &
sleep 1 ; load_wait 3 <command4> &
...
I then launch those via "system" from inside awk and I'm done. The commands tend to be in muscle memory.