2 ms·
The sure-fire one-liner Bash way to iterate over files with arbitrary filenames, with any number of files (0 to millions) is this: $ find $CRITERIA -print0
by m104 14y ago
The sure-fire one-liner Bash way to iterate over files with arbitrary filenames, with any number of files (0 to millions) is this:
$ find $CRITERIA -print0 | xargs -0 -r -n $BATCH_SIZE $COMMAND
What this does is use 'find' to find the file and directory names (with your supplied CRITERIA), send them over to xargs with no funny filename parsing, run those filenames as arguments to COMMAND, batched up to BATCH_SIZE at a time (I usually use 100-500), and ignoring empty batches.
Usually COMMAND would have '--' at the end to prevent filenames from being parsed as arguments, but not all commands need that treatment. If COMMAND doesn't take the filename list as the last argument(s), it can be fixed with the '-i' option to xargs.
The BATCH_SIZE part here is important in many cases. Since large numbers of filenames will break the command line arguments size limit to COMMAND, a reasonable BATCH_SIZE will prevent thousands or millions of filenames from breaking your script. Also, sometimes you may want to run the files through COMMAND one at a time (BATCH_SIZE of 1).
Finally, the '-r' option, which I don't see mentioned often enough, tells xargs not to be derpy and run COMMAND with an empty argument list. Seriously, '-r' is short for '--no-run-if-empty'.
For anything else, a nice scripting language is the only way to fly.