3 ms·
> rsync over ssh works perfectly I use rsync+ssh a lot, and it doesn't exactly work perfectly on long-distance 10-100 gigabit networks: ssh has a buffering pro
by wumpus 5y ago
> rsync over ssh works perfectly
I use rsync+ssh a lot, and it doesn't exactly work perfectly on long-distance 10-100 gigabit networks: ssh has a buffering problem at high bandwidths which the maintainers won't accept patches to fix.
- adrian_b 5y agoThat is a limitation of ssh. Using scp or sftp would limit the transfer speed to a value several times lower than rsync in any case. On short distance 10 Gb/s Ethernet I did not see problems, but it does not surprise me that on long-distance links they appear. In such cases, if full speed is desired, it is likely that ssh must be replaced with either IPsec or a tunnel over TLS.
- Datagenerator 5y agoTried rclone yet? Does it all, parallel and fast.
- betaby 5y agoWhat back-end do you use? Also on top of that back-end do use type = crypt, directory_name_encryption = true and so on? I tied various options and found rclone were slow for me.
- wumpus 5y agoscp has exactly the same limitations as ssh, it's the same code. I don't know anyone in the HPC world who uses "IPsec or a tunnel over TLS".
- effie 5y ago> ssh has a buffering problem at high bandwidths This is interesting, can this be the reason for why playing 720p video in Firefox run with ssh and X11 forwarding is fine bandwidth-wise (easily 1Gb/s) but it causes responsiveness to mouse/keyboard actions to go into abysmal 5-30s territory? Any references to the ssh buffering problem out there?