3 ms·

I have, admittedly old and very vague, memories of people talking about rsync being "hard on networks" or "dealing poorly with congestion." I'd put good odds th

by rancor 6y ago

I have, admittedly old and very vague, memories of people talking about rsync being "hard on networks" or "dealing poorly with congestion." I'd put good odds that this bug is why those statements existed.

bonzini 6y ago
The bug requires transferring over 2GB of data without reading from the socket, so it's unlikely; also, a hang is the opposite of being hard on the network. ;) However the uncommon characteristics of rsync traffic is probably why some congestion control algorithms may not deal well with rsync.
jandrese 6y ago
This seems to be the opposite. You only see it when transferring titanic amounts of data over a pristine connection. If your network had congestion you wouldn't trigger this bug. But this also explains a bit why rsync is "hard on networks". Most bulk data transfers end up with breaks in the data that give more breathing room to other protocols. Not rsync, it tries as hard as it can to keep the pipe full 100% of the time, making it hard for other TCP slow starters to get a foothold.
- tinus_hn 6y ago
  BitTorrent does the same thing and used to be a lot more common, just typically not between hosts close to each other.
  - M2Ys4U 6y ago
    TCP-based bittorrent, yes. The uTP protocol (which has an IETF version called LEDBAT) was specifically designed to be used for background transmission, and it should slow itself down if there are competing TCP flows.
- breatheoften 6y ago
  rsync is hard on networks??????????? Is there any truth to this? I find it hard to believe -- most of the time rsync is tunneled over ssh which seems well enough abstracted from an optimal traffic generation mechanism that i would seriously doubt it's able to outcompete other programs for network resources in a meaningful way ... perhaps this observation evolved because there are a lot of networks that have traffic shaping rules for ssh? unfortunate effects of traffic shaping rules for ssh + low bandwidth connection + rsyncs happening over ssh + an administrator logged into an ssh port via the low bandwidth link could maybe produce this observed (but non-sensical?) correlation?
- Dylan16807 6y ago
  One big fat TCP connection isn't so bad on networks. Especially the default behavior of slowly creeping up in speed until it loses a packet, then dropping back down. As I understand it significant factor in getting this bug to happen is that you're sending tons of data but in a way that's limited by the source.

by rancor 6y ago

bonzini 6y ago

The bug requires transferring over 2GB of data without reading from the socket, so it's unlikely; also, a hang is the opposite of being hard on the network. ;) However the uncommon characteristics of rsync traffic is probably why some congestion control algorithms may not deal well with rsync.

jandrese 6y ago

This seems to be the opposite. You only see it when transferring titanic amounts of data over a pristine connection. If your network had congestion you wouldn't trigger this bug. But this also explains a bit why rsync is "hard on networks". Most bulk data transfers end up with breaks in the data that give more breathing room to other protocols. Not rsync, it tries as hard as it can to keep the pipe full 100% of the time, making it hard for other TCP slow starters to get a foothold.

tinus_hn 6y ago
BitTorrent does the same thing and used to be a lot more common, just typically not between hosts close to each other.
- M2Ys4U 6y ago
  TCP-based bittorrent, yes. The uTP protocol (which has an IETF version called LEDBAT) was specifically designed to be used for background transmission, and it should slow itself down if there are competing TCP flows.
breatheoften 6y ago
rsync is hard on networks??????????? Is there any truth to this? I find it hard to believe -- most of the time rsync is tunneled over ssh which seems well enough abstracted from an optimal traffic generation mechanism that i would seriously doubt it's able to outcompete other programs for network resources in a meaningful way ... perhaps this observation evolved because there are a lot of networks that have traffic shaping rules for ssh? unfortunate effects of traffic shaping rules for ssh + low bandwidth connection + rsyncs happening over ssh + an administrator logged into an ssh port via the low bandwidth link could maybe produce this observed (but non-sensical?) correlation?
Dylan16807 6y ago
One big fat TCP connection isn't so bad on networks. Especially the default behavior of slowly creeping up in speed until it loses a packet, then dropping back down. As I understand it significant factor in getting this bug to happen is that you're sending tons of data but in a way that's limited by the source.