4 ms·

The one that I find repeatedly useful: find . -name <some pattern> | { while read F; do <do something to file F>; done; } This lets you do things of almost an

by t43562 5y ago

The one that I find repeatedly useful: find . -name <some pattern> | { while read F; do <do something to file F>; done; } This lets you do things of almost any level of complexity to files or data listed as input and it doesn't require temporary files so you can process a lot of things. e.g. you might use sed -i to alter the files in some way or you might find media files of some kind into ffmpeg to convert them.

chasil 5y ago
If you really had a lot of ffmpeg work to do, then GNU xargs can let you run the jobs in parallel, one for each of your available CPUs. You can even adjust the number of processes up or down by sending the xargs parent the appropriate signals. https://www.linuxjournal.com/content/parallel-shells-xargs-utilize-all-your-cpu-cores-unix-and-windows https://www.linuxjournal.com/content/parallel-shells-xargs-u...
- t43562 5y ago
  xargs is a fantastic tool - there's another similar one, think called GNU parallel which is worth a look: https://www.gnu.org/software/parallel/ https://www.gnu.org/software/parallel/
geocrasher 5y ago
You can use another construct built into find that negates the need for a for loop: find . -name foo -exec do something to {} \; For example: find . type f -exec chmod 644 {} \; find . -size +1G -exec ls -lh {} \; |awk '{ print $5" "$9 }'
- t43562 5y ago
  Yes, I think most of the time this is all you need. I just like the for loop because I can do filename manipulations in the loop and the input to the loop doesn't always have to be find - it can be "grep <something> -l ./" for example - so I use it even if it's verbose because it's only one thing to remember. Perhaps that is a problem with bash and the tools - there are so many features that take years to discover. Every few weeks I come across a new one.
cma 5y ago
Unfortunately filenames can have newlines so this isn't safe for robust scripts.
- t43562 5y ago
  One can make it safe if it's worth the effort. In practice I don't ever come across such files but if I was trying to write a script that would work everywhere I would have to use find -print0 | sed -r 's#\r#\\r#g;s#\n#\\n#g;s#\x0#\n#g; ... and then I would have to remember to use the filenames in such a way that my escaped newlines would be converted back to real ones at the point where I accessed the file Newlines in filenames is one of those things that one would personally try to avoid purely because of the problems it can cause.
  - cma 5y ago
    It is fine for a personal script where you know your files don't have newlines, but for a robust script as part of a package running on other people's systems you'd have to take it into account.

by t43562 5y ago

chasil 5y ago

If you really had a lot of ffmpeg work to do, then GNU xargs can let you run the jobs in parallel, one for each of your available CPUs. You can even adjust the number of processes up or down by sending the xargs parent the appropriate signals. https://www.linuxjournal.com/content/parallel-shells-xargs-utilize-all-your-cpu-cores-unix-and-windows https://www.linuxjournal.com/content/parallel-shells-xargs-u...

t43562 5y ago
xargs is a fantastic tool - there's another similar one, think called GNU parallel which is worth a look: https://www.gnu.org/software/parallel/ https://www.gnu.org/software/parallel/

geocrasher 5y ago

You can use another construct built into find that negates the need for a for loop: find . -name foo -exec do something to {} \; For example: find . type f -exec chmod 644 {} \; find . -size +1G -exec ls -lh {} \; |awk '{ print $5" "$9 }'

t43562 5y ago
Yes, I think most of the time this is all you need. I just like the for loop because I can do filename manipulations in the loop and the input to the loop doesn't always have to be find - it can be "grep <something> -l ./" for example - so I use it even if it's verbose because it's only one thing to remember. Perhaps that is a problem with bash and the tools - there are so many features that take years to discover. Every few weeks I come across a new one.

cma 5y ago

Unfortunately filenames can have newlines so this isn't safe for robust scripts.

t43562 5y ago
One can make it safe if it's worth the effort. In practice I don't ever come across such files but if I was trying to write a script that would work everywhere I would have to use find -print0 | sed -r 's#\r#\\r#g;s#\n#\\n#g;s#\x0#\n#g; ... and then I would have to remember to use the filenames in such a way that my escaped newlines would be converted back to real ones at the point where I accessed the file Newlines in filenames is one of those things that one would personally try to avoid purely because of the problems it can cause.
- cma 5y ago
  It is fine for a personal script where you know your files don't have newlines, but for a robust script as part of a package running on other people's systems you'd have to take it into account.