One fun thing is that xargs supports parallelization. -P max-procs Run up to max-procs processes at a time; the default is 1. If max-procs is 0, xa

One fun thing is that xargs supports parallelization. -P max-procs Run up to max-procs processes at a time; the default is 1. If max-procs is 0, xargs will run as many processes as possible at a time. Use the -n option with -P; otherwise chances are that only one exec will be done. -n max-args Use at most max-args arguments per command line. Fewer than max-args arguments will be used if the size (see the -s option) is exceeded, unless the -x option is given, in which case xargs will exit. So here is a "map/reduce" job which takes the log files in a directory and processes them in parallel on eight CPUs, then combines the results. find . -name "*.log" | sort | xargs -n 1 -P 8 agg_stats.py | sort | merge_periods.py