3 ms·

What are the conditions leading to extreme packet loss in layers 2&3 in the first place? I can imagine noisy RF, industrial, congested links, new queueing at t

by ggm 2y ago

What are the conditions leading to extreme packet loss in layers 2&3 in the first place? I can imagine noisy RF, industrial, congested links, new queueing at the extremes in densely loaded switches, but the thing is: usually out there are strategies to reduce the congestion. External noise, factory/industrial/adversarial, sure. This is going to exist.

AdamJacobMuller 2y ago
At 0-1%, basically anything. Your wifi and home network probably are closer to 1% than 0%. at 1-10%, you're probably on some kind of shared connection (CMTS or GPON with TDMA) and your provider has an overloaded network design (and you're at the end of the upgrade/split queue). at anything above 10% you're in the realm of weird broken stuff. Congestion on the internet between major providers is a thing but is far less common than it was a decade ago, it does still happen though. Major providers who have NECMP or NLAG links inside their backbones or between providers where one of the links breaks in a way which doesn't remove it from the ECMP/LAG and suddenly you drop 1/N of the traffic (where N might be 2 == 50% loss). More common was finding LAG/ECMP where 1/N of the links was oversubscribed but the others were fine due to unequal traffic distribution. > noisy RF, industrial Pretty uncommon in my experience as most of these environments are not as cost optimized and know already that they will be in weird RF/electrical environments so they just use wired ethernet (and even shielded cable for really janky situations). While there are strategies to reduce congestion on backbone links, they are not commonly implemented and even less commonly implemented well and sometimes implemented intentionally poorly.
- ggm 2y ago
  There isn't much point in wierd algo solutions to 1% packetloss, and most people are living under 5% most of the time. I would say on an 80/20 class decision most of us either have layer2 loss in the link to the provider, and then not much loss, or have some corner case of congestion like zoom inter-continentally. Remember, 75% of content now comes from a DC within 1 AS hop of your ISP. For non-western economies reliant on mobile IP, there is a level of loss and congestion being seen in the radio packet layer but that layer is usually dealing with it. Starlink has worked out how to manage it's TCP loss issues despite steering dishy between sats in orbit in a 15 second timeslot model. I'm not trying to dismiss this work: I don't understand the use-case because from my perspective (admittedly in the western economy, but in asia and exposed to the other kind of internet for less developed places) this level of sustained packetloss isn't usual, And when it is, the link layer is usually doing something like FEC to take care of it.
  - AdamJacobMuller 2y ago
    I think the use-case for the author is "this is cool" which is the best reason for anything to exist.
  - zamalek 2y ago
    Even though 1% loss is relatively common, it is a catastrophic amount of loss, and goes to show the miracle of TCP. 1% loss does not mean 1% less goodput; it means substantially more than 1%. TCP's (and QUIC, and what-have-you) control packets are also subject to that loss, as are individual packet fragments - the probability of retransmissions explodes.
vasilvv 2y ago
Generally, L2 networks are engineered with the assumption that they will carry TCP, and TCP performs really poorly with high loss rates (depends on the specific congestion control used, but the boundary can be anywhere between 1% and 25%), so they try to make sure on L2 level that losses are minimal. There are some scenarios in which a network can be engineered around high loss rates (e.g. some data center networks), but those don't use TCP, at least with traditional loss recovery. Error correction codes on L4 level are generally only useful for very low latency situations, since if you can wait for one RTT, you can just have the original sender retransmit the exact packets that got lost, which is inherently more efficient than any ECC.

by ggm 2y ago

AdamJacobMuller 2y ago

At 0-1%, basically anything. Your wifi and home network probably are closer to 1% than 0%. at 1-10%, you're probably on some kind of shared connection (CMTS or GPON with TDMA) and your provider has an overloaded network design (and you're at the end of the upgrade/split queue). at anything above 10% you're in the realm of weird broken stuff. Congestion on the internet between major providers is a thing but is far less common than it was a decade ago, it does still happen though. Major providers who have NECMP or NLAG links inside their backbones or between providers where one of the links breaks in a way which doesn't remove it from the ECMP/LAG and suddenly you drop 1/N of the traffic (where N might be 2 == 50% loss). More common was finding LAG/ECMP where 1/N of the links was oversubscribed but the others were fine due to unequal traffic distribution. > noisy RF, industrial Pretty uncommon in my experience as most of these environments are not as cost optimized and know already that they will be in weird RF/electrical environments so they just use wired ethernet (and even shielded cable for really janky situations). While there are strategies to reduce congestion on backbone links, they are not commonly implemented and even less commonly implemented well and sometimes implemented intentionally poorly.

ggm 2y ago
There isn't much point in wierd algo solutions to 1% packetloss, and most people are living under 5% most of the time. I would say on an 80/20 class decision most of us either have layer2 loss in the link to the provider, and then not much loss, or have some corner case of congestion like zoom inter-continentally. Remember, 75% of content now comes from a DC within 1 AS hop of your ISP. For non-western economies reliant on mobile IP, there is a level of loss and congestion being seen in the radio packet layer but that layer is usually dealing with it. Starlink has worked out how to manage it's TCP loss issues despite steering dishy between sats in orbit in a 15 second timeslot model. I'm not trying to dismiss this work: I don't understand the use-case because from my perspective (admittedly in the western economy, but in asia and exposed to the other kind of internet for less developed places) this level of sustained packetloss isn't usual, And when it is, the link layer is usually doing something like FEC to take care of it.
- AdamJacobMuller 2y ago
  I think the use-case for the author is "this is cool" which is the best reason for anything to exist.
- zamalek 2y ago
  Even though 1% loss is relatively common, it is a catastrophic amount of loss, and goes to show the miracle of TCP. 1% loss does not mean 1% less goodput; it means substantially more than 1%. TCP's (and QUIC, and what-have-you) control packets are also subject to that loss, as are individual packet fragments - the probability of retransmissions explodes.

vasilvv 2y ago

Generally, L2 networks are engineered with the assumption that they will carry TCP, and TCP performs really poorly with high loss rates (depends on the specific congestion control used, but the boundary can be anywhere between 1% and 25%), so they try to make sure on L2 level that losses are minimal. There are some scenarios in which a network can be engineered around high loss rates (e.g. some data center networks), but those don't use TCP, at least with traditional loss recovery. Error correction codes on L4 level are generally only useful for very low latency situations, since if you can wait for one RTT, you can just have the original sender retransmit the exact packets that got lost, which is inherently more efficient than any ECC.