3 ms·
Not being a jerk, but what does parallel bring to the table that xargs doesn't? find . -type f -name | xargs -I {} -n 1 -P 5 process_file {}
by coredog64 10y ago
Not being a jerk, but what does parallel bring to the table that xargs doesn't?
find . -type f -name | xargs -I {} -n 1 -P 5 process_file {}
- perturbation 10y agoYou can have remote workers with parallel (not sure if you can do that with xargs or not). parallel --progress --wd '.' -S '4/fourworkers@blehworker.org,8/eightworkers@eightworkers.org,2/:' "./complicated_command --param1={1} --param2={2} ::: $(some_shellcommand_to_generate_param1_values.sh) ::: $(some_shellcommand_to_generate_param2_values.sh) It's very flexible, become my go-to for speeding up parallelizable work. Next step up would be Celery or Spark or something.
- ole_tange 10y agoBigger list: https://www.gnu.org/software/parallel/man.html#DIFFERENCES-BETWEEN-xargs-AND-GNU-Parallel https://www.gnu.org/software/parallel/man.html#DIFFERENCES-B...