3 ms·
Wouldn't lots of streams speed up transfers of thousands of small files?
by Saris 8mo ago
Wouldn't lots of streams speed up transfers of thousands of small files?
- digiown 8mo agoIf the application handles them serially, then yeah. But one can imagine the application opening files in threads, buffering them, and then finally sending it at full speed, so in that sense it is an application issue. If you truly have millions of small files, you're more likely to be bottlenecked by disk IO performance rather than application or network, though. My primary use case for ssh streams is zfs send, which is mostly bottlenecked by ssh itself.
- catdog 8mo agoIt's an application issue but implementation wise it's probably way more straightforward to just open a separate network connection per thread.