3 ms·
the biggest weakness of rsync is that it's single-threaded. Single-thread-copy with millions of small files is painfully slow, Rsync is as good as it can be bu
by paulmd 5y ago
the biggest weakness of rsync is that it's single-threaded. Single-thread-copy with millions of small files is painfully slow, Rsync is as good as it can be but you just need more threads.
I think that's what a sibling is getting into with the "better to tar files and send them over ssh in some cases" thing. And yes, you can hack it in after-the-fact with xargs/etc but it's clunky compared to just having native multithreading like rclone/etc.
- jimpudar 5y agoIf you use GNU Parallel, there's a fairly easy way to parallelize [1]. That being said, I hadn't heard of rclone - thanks for mentioning it, it looks amazing. I'll definitely be trying this out for my use cases... [1] http://www.gnu.org/software/parallel/man.html#example-parallelizing-rsync http://www.gnu.org/software/parallel/man.html#example-parall...
- robertlagrant 5y agoWow, Parallel also looks awesome. Never seen it before.