2 ms·
Forking in the background is one way, you can also use GNU "parallel" ("apt-get install parallel" for those on Debian and the likes). For example I've got all
by TacticalCoder 5y ago
Forking in the background is one way, you can also use GNU "parallel" ("apt-get install parallel" for those on Debian and the likes).
For example I've got all my CD ripped to FLAC files, but my car only takes mp3 or wav... So I did a batch convert of FLAC to mp3, making sure to put all cores at work by piping the output of "find" into "parallel".
Some commands also allows to directly parallelize (like, say, "make -j 16 ..." to build using 16 cores/hyperthreads).
- chockchocschoir 5y agoAnother tip is using wget2 instead of wget if you're mirroring a site (but this is more I/O tip than computationally heavy) https://gitlab.com/gnuwget/wget2/-/wikis/home https://gitlab.com/gnuwget/wget2/-/wikis/home Sadly, wget2 doesn't support WARC last time I checked, but wget2 comes with a `--max-threads` parameter that together with `--mirror` and `--tries` makes it trivial to mirror even the slowest websites out there. Edit: your parallel to `parallel` made me think of wget2 as I often see scripts that use `parallel` together with `wget` when `wget2` can be much better to use alone instead of pairing the two. Just wanted to add some context.