Another tip is using wget2 instead of wget if you're mirroring a site (but this is more I/O tip than computationally heavy) https://gitlab.com/gnuwget/wget2/-/w

Another tip is using wget2 instead of wget if you're mirroring a site (but this is more I/O tip than computationally heavy) https://gitlab.com/gnuwget/wget2/-/wikis/home https://gitlab.com/gnuwget/wget2/-/wikis/home Sadly, wget2 doesn't support WARC last time I checked, but wget2 comes with a `--max-threads` parameter that together with `--mirror` and `--tries` makes it trivial to mirror even the slowest websites out there. Edit: your parallel to `parallel` made me think of wget2 as I often see scripts that use `parallel` together with `wget` when `wget2` can be much better to use alone instead of pairing the two. Just wanted to add some context.