3 ms·

why would you ever pipe find into xargs instead of calling -exec? find -exec the '{}' iterated command ';'

by em-bee 3mo ago

gumby 3mo ago
Because xargs is faster. Exec will invoke the command once per matching file (which is sometimes what you want, of course)! While xargs will accumulate a bunch of file names, then when it as n names will invoke the command with those names, while continuing to accumulate names until n is reached or the pipe closes. The size of n depends on the system, but is usually at least a thousand.
- em-bee 3mo ago
  the example as given does not accumulate, so that is what i worked with. find -exec command '{}' '+' accumulates file arguments too. the only advantage of xargs is that you can tell it how many arguments to accumulate,̶ ̶t̶h̶e̶ ̶d̶o̶w̶n̶s̶i̶d̶e̶ ̶o̶f̶ ̶x̶a̶r̶g̶s̶ ̶i̶s̶ ̶t̶h̶a̶t̶ ̶y̶o̶u̶ ̶h̶a̶v̶e̶ ̶t̶o̶ ̶s̶p̶e̶c̶i̶f̶y̶ ̶t̶h̶e̶ ̶n̶u̶m̶b̶e̶r̶,̶ ̶w̶h̶e̶r̶e̶a̶s̶ ̶f̶i̶n̶d̶ ̶j̶u̶s̶t̶ ̶f̶i̶t̶s̶ ̶a̶s̶ ̶m̶a̶n̶y̶ ̶a̶s̶ ̶i̶t̶ ̶c̶a̶n̶.̶
  - unsnap_biceps 3mo ago
    xargs will fit as many as it can in 128 KiB or the system limit, which is smaller, so in practice, it's almost always pretty similar default packing
    - em-bee 3mo ago
      you are right, i forgot to check the default. fixed my comment.
  - xorcist 3mo ago
    The problem with find is that you can't specify the maximum tolerated command line lengths, and find hasn't historically been very smart about it (it just had a compiled-in value). Apparently this is fixed in GNU find, but for older systems and other platforms that may still be an issue. Another feature missing in find that xargs has is the maximum processes to start at a time. find will run the commands in sequence, but in many situations you really want to run a bunch in parallel.
    - em-bee 3mo ago
      good points that i wasn't aware of, thank you. though personally i rarely start huge operations that need to be parallelized so i prefer simplicity over speed.
- ggm 3mo ago
  The prior has a point because 98.89% of the time I type xargs -n 1 -0 and we are deep in the useless pipe argument which Rob Pike amongst others has rehearsed well. I do it because I do it, just like why I use egrep and sed in pipes along with awk.
t-3 3mo ago
Leah Neukirchen's lr and xe are very nice as a find and xargs replacement (although lr's test flag is way too complex). lr *.file | xe -s ' ... ' is a really great pattern for iterative scripting and hard to get wrong.

why would you ever pipe find into xargs instead of calling -exec? find -exec the '{}' iterated command ';'

by em-bee 3mo ago

why would you ever pipe find into xargs instead of calling -exec? find -exec the '{}' iterated command ';'

gumby 3mo ago

Because xargs is faster. Exec will invoke the command once per matching file (which is sometimes what you want, of course)! While xargs will accumulate a bunch of file names, then when it as n names will invoke the command with those names, while continuing to accumulate names until n is reached or the pipe closes. The size of n depends on the system, but is usually at least a thousand.

em-bee 3mo ago
the example as given does not accumulate, so that is what i worked with. find -exec command '{}' '+' accumulates file arguments too. the only advantage of xargs is that you can tell it how many arguments to accumulate,̶ ̶t̶h̶e̶ ̶d̶o̶w̶n̶s̶i̶d̶e̶ ̶o̶f̶ ̶x̶a̶r̶g̶s̶ ̶i̶s̶ ̶t̶h̶a̶t̶ ̶y̶o̶u̶ ̶h̶a̶v̶e̶ ̶t̶o̶ ̶s̶p̶e̶c̶i̶f̶y̶ ̶t̶h̶e̶ ̶n̶u̶m̶b̶e̶r̶,̶ ̶w̶h̶e̶r̶e̶a̶s̶ ̶f̶i̶n̶d̶ ̶j̶u̶s̶t̶ ̶f̶i̶t̶s̶ ̶a̶s̶ ̶m̶a̶n̶y̶ ̶a̶s̶ ̶i̶t̶ ̶c̶a̶n̶.̶
- unsnap_biceps 3mo ago
  xargs will fit as many as it can in 128 KiB or the system limit, which is smaller, so in practice, it's almost always pretty similar default packing
  - em-bee 3mo ago
    you are right, i forgot to check the default. fixed my comment.
- xorcist 3mo ago
  The problem with find is that you can't specify the maximum tolerated command line lengths, and find hasn't historically been very smart about it (it just had a compiled-in value). Apparently this is fixed in GNU find, but for older systems and other platforms that may still be an issue. Another feature missing in find that xargs has is the maximum processes to start at a time. find will run the commands in sequence, but in many situations you really want to run a bunch in parallel.
  - em-bee 3mo ago
    good points that i wasn't aware of, thank you. though personally i rarely start huge operations that need to be parallelized so i prefer simplicity over speed.
ggm 3mo ago
The prior has a point because 98.89% of the time I type xargs -n 1 -0 and we are deep in the useless pipe argument which Rob Pike amongst others has rehearsed well. I do it because I do it, just like why I use egrep and sed in pipes along with awk.

t-3 3mo ago

Leah Neukirchen's lr and xe are very nice as a find and xargs replacement (although lr's test flag is way too complex). lr *.file | xe -s ' ... ' is a really great pattern for iterative scripting and hard to get wrong.