3 ms·

> The main problem you're talking of is head of line blocking due to packet loss. But packet loss as a congestion signal is nowhere near as common as people thi

> The main problem you're talking of is head of line blocking due to packet loss. But packet loss as a congestion signal is nowhere near as common as people think Surely packet loss due to poor signal quality is rather common over mobile networks and that packet loss still affects TCP's congestion window. Admittedly anecdotal, but I just connected to a 5G network with low signal strength and it certainly seems to be the case.

pi-e-sigma 3y ago
The parent is just moving goal posts. The whole idea behind multiplexing data streams inside a single TCP connection was that in case of a packet loss you don't lose all your streams. But it doesn't work in practice which is not really surprising when you think about it. When you have multiple TCP connections it's less likely that all of them will get reset due to connectivity issues. Whereas with data multiplexing when your single TCP connection is reset all your data flow stops and need to be restarted.
- kikimora 3y ago
  A problem with radio signal or with a wire would affect all TCP connections at the same time. It does not matter if it is one or many, the outcome will be the same. I believe in real life this is a majority of cases. A problem affecting just one TCP connection out of many on the same link must be related to the software on the other side, not network itself.
jsnell 3y ago
It isn't. Or at least wasn't back in the UMTS / early LTE era that's being discussed; I got out of that game before 5G. The base stations and terminals are constantly monitoring the signal quality and adjusting the error correction rate. A bad signal will mean that there's more error correction overhead, and that's why the connection is slower overall. Second,the radio protocol doesn't treat data transmissions as a one-and-done deal. The protocols are built on the assumption of a high rate of unrecoverable transmission errors. Those rates would be way too high for TCP to be able to function, so retransmissions are instead baked in at the physical protocol level. The cellular base station will basically buffer the data until it's been reliably delivered or until the client moves to a different cell. And crucially, not only is the physical protocol reliable but it's also in-sequences. A packet that wasn't received successfully shows up just as one (radio) roundtrip latency blip during which no data at all shows up at the client, not as packet loss or reordering that would be visible to the TCP stack on either side. Other error cases you'd get are: - Massive amounts of queueing, with the queues being per-user rather than per-base station (we measured up to a minute of queueing in testing). The queue times would translate directly to latency, completely dominating any other component. - A catastrophic loss of all in-flight packets at once, which we believed was generally caused by handover from one cell to another.
- pi-e-sigma 3y ago
  Everyone who is using a phone knows that what you are saying is not true. Otherwise we would not experience dropped calls, connection resets and mobile data being unavailable. Mobile networks are unreliable and you can't paper it over with some magic on TCP or HTTP2/3 level. EDIT: better yet, anyone can just use network tools on their smartphone to see for themselves that mobile networks do drop TCP packets, UDP packets and ICMP packets very freely. Just check yourself!
  - jsnell 3y ago
    Huh? I'm not talking about papering over it on the TCP or HTTP/2 level. I'm talking about the actual physical radio protocols, and my message could not be more explicit about it. If you don't understand something, it'd be more productive to ask questions than just make shit up like that.
    - pi-e-sigma 3y ago
      You made a claim that packet loss in mobile networks is not a common occurrence. This claim is patently wrong and anyone with a smartphone can see for themselves.
      - jsnell 3y ago
        In reality it's quite hard for somebody to observe that themselves using just their smartphone. The only way they can do it is by getting a packet trace, which they won't have the permissions to capture on the phone, nor the skill to interpret. (Ideally they'd want to get packet traces from multiple points in the network to understand where the anomalies are happening, but that's even harder.) In principle you could observe it from some kind of OS level counters that aren't ACLd, but in practice the counters are not reliable enough for that. Now, the things like "calls dropping" or "connections getting reset" that you're calling out have nothing to do with packet loss. It's pretty obvious that you're not very technical and think that all error conditions are just the same thing and you can just mix them together. But what comes out is just technobabble.
rstuart4133 3y ago
> Surely packet loss due to poor signal quality is rather common over mobile networks and that packet loss still affects TCP's congestion window. Two points: 1. It's not commonly realised the TCP is terrible on lossy networks, where terrible means gets less than 10% of the potential throughput. It only becomes apparent when you try to use TCP over a lossy network of course, and most real networks we use aren't lossy. Engineers who try to use TCP over lossy networks end up replacing it with something else. FWIW, the problem is TCP uses packet loss as a congestion signal. It handles congestion pretty well by backing off. But packet loss can also mean the packet was actually lost. The right responses in that case are to reduce the packet size and/or increase error correction, but _not_ decrease your transmission rate. Thus two responses to the same signal conflict. 2. Because of that, the layer two networks the internet uses have evolved to have really low error rates, which is why most people don't experience TCP's problems in that area. As it happens just about any sort of wireless has really high error rates, so they have to mask it. And they do, by having lots of ECC and doing their own ACK/NAKs. This might create lots of fluctuations in available bandwidth - but that is what TCP is good at handling. By the by, another reason we have come to depend on really low error rates on layer 2. That's because TCP's error detection is poor. It lets roughly one bad packet through in every 10,000. (Adler32 is very poor on small packets.) You can send 100,000 packets a second at 1Gb/sec, so you need to keep the underlying error rate very low to ensure the backup you are sending to Backblaze isn't mysteriously corrupted a few times a year. <rant>IMO, we should have switched to 64 bit CRC's decades ago.</rant>

pi-e-sigma 3y ago

The parent is just moving goal posts. The whole idea behind multiplexing data streams inside a single TCP connection was that in case of a packet loss you don't lose all your streams. But it doesn't work in practice which is not really surprising when you think about it. When you have multiple TCP connections it's less likely that all of them will get reset due to connectivity issues. Whereas with data multiplexing when your single TCP connection is reset all your data flow stops and need to be restarted.

kikimora 3y ago
A problem with radio signal or with a wire would affect all TCP connections at the same time. It does not matter if it is one or many, the outcome will be the same. I believe in real life this is a majority of cases. A problem affecting just one TCP connection out of many on the same link must be related to the software on the other side, not network itself.

jsnell 3y ago

It isn't. Or at least wasn't back in the UMTS / early LTE era that's being discussed; I got out of that game before 5G. The base stations and terminals are constantly monitoring the signal quality and adjusting the error correction rate. A bad signal will mean that there's more error correction overhead, and that's why the connection is slower overall. Second,the radio protocol doesn't treat data transmissions as a one-and-done deal. The protocols are built on the assumption of a high rate of unrecoverable transmission errors. Those rates would be way too high for TCP to be able to function, so retransmissions are instead baked in at the physical protocol level. The cellular base station will basically buffer the data until it's been reliably delivered or until the client moves to a different cell. And crucially, not only is the physical protocol reliable but it's also in-sequences. A packet that wasn't received successfully shows up just as one (radio) roundtrip latency blip during which no data at all shows up at the client, not as packet loss or reordering that would be visible to the TCP stack on either side. Other error cases you'd get are: - Massive amounts of queueing, with the queues being per-user rather than per-base station (we measured up to a minute of queueing in testing). The queue times would translate directly to latency, completely dominating any other component. - A catastrophic loss of all in-flight packets at once, which we believed was generally caused by handover from one cell to another.

pi-e-sigma 3y ago
Everyone who is using a phone knows that what you are saying is not true. Otherwise we would not experience dropped calls, connection resets and mobile data being unavailable. Mobile networks are unreliable and you can't paper it over with some magic on TCP or HTTP2/3 level. EDIT: better yet, anyone can just use network tools on their smartphone to see for themselves that mobile networks do drop TCP packets, UDP packets and ICMP packets very freely. Just check yourself!
- jsnell 3y ago
  Huh? I'm not talking about papering over it on the TCP or HTTP/2 level. I'm talking about the actual physical radio protocols, and my message could not be more explicit about it. If you don't understand something, it'd be more productive to ask questions than just make shit up like that.
  - pi-e-sigma 3y ago
    You made a claim that packet loss in mobile networks is not a common occurrence. This claim is patently wrong and anyone with a smartphone can see for themselves.
    - jsnell 3y ago
      In reality it's quite hard for somebody to observe that themselves using just their smartphone. The only way they can do it is by getting a packet trace, which they won't have the permissions to capture on the phone, nor the skill to interpret. (Ideally they'd want to get packet traces from multiple points in the network to understand where the anomalies are happening, but that's even harder.) In principle you could observe it from some kind of OS level counters that aren't ACLd, but in practice the counters are not reliable enough for that. Now, the things like "calls dropping" or "connections getting reset" that you're calling out have nothing to do with packet loss. It's pretty obvious that you're not very technical and think that all error conditions are just the same thing and you can just mix them together. But what comes out is just technobabble.

rstuart4133 3y ago

> Surely packet loss due to poor signal quality is rather common over mobile networks and that packet loss still affects TCP's congestion window. Two points: 1. It's not commonly realised the TCP is terrible on lossy networks, where terrible means gets less than 10% of the potential throughput. It only becomes apparent when you try to use TCP over a lossy network of course, and most real networks we use aren't lossy. Engineers who try to use TCP over lossy networks end up replacing it with something else. FWIW, the problem is TCP uses packet loss as a congestion signal. It handles congestion pretty well by backing off. But packet loss can also mean the packet was actually lost. The right responses in that case are to reduce the packet size and/or increase error correction, but _not_ decrease your transmission rate. Thus two responses to the same signal conflict. 2. Because of that, the layer two networks the internet uses have evolved to have really low error rates, which is why most people don't experience TCP's problems in that area. As it happens just about any sort of wireless has really high error rates, so they have to mask it. And they do, by having lots of ECC and doing their own ACK/NAKs. This might create lots of fluctuations in available bandwidth - but that is what TCP is good at handling. By the by, another reason we have come to depend on really low error rates on layer 2. That's because TCP's error detection is poor. It lets roughly one bad packet through in every 10,000. (Adler32 is very poor on small packets.) You can send 100,000 packets a second at 1Gb/sec, so you need to keep the underlying error rate very low to ensure the backup you are sending to Backblaze isn't mysteriously corrupted a few times a year. <rant>IMO, we should have switched to 64 bit CRC's decades ago.</rant>