3 ms·

TCP has major issues in low-latency environments. It has too many timers and they are too coarse for 50us RTTs and 100gbps bandwidth. QUIC as a transport for gR

by romed 8y ago

TCP has major issues in low-latency environments. It has too many timers and they are too coarse for 50us RTTs and 100gbps bandwidth. QUIC as a transport for gRPC can solve a lot of this. What we know for sure today is that Linux distributions, out of the box, are garbage for TCP in datacenters. You can tune some of it away, if you happen to know where the knobs are, but some things like microsecond packet time stamps you have to patch. Switching to a UDP transport means you don’t have to argue with Linux kernel maintainers in order to make progress.

ivoras 8y ago
First actually useful and technical answer I've seen in this thread. But... which use cases of HTTP need us timers? It's not like gmail or other bloated web apps will load faster on my phone because of QUIC?
- romed 8y ago
  The timers are in TCP not HTTP. TCP only ACKs every other packet. If the stream pauses after an odd number of packets, the sender has to just wait for the receiver to perform a delayed ACK, for which there is a timer on both sides. The sender infers that the last packet has been lost if the ACK doesn't arrive after a while. The sender has a thing called the retransmission timeout (RTO) and the receiver's delayed ACK timeout (ATO) sets a lower bound on the RTO. Modern TCP stacks have a thing called the tail loss probe that just sends a retransmit early (before the RTO) if the stream has stopped at an odd boundary. This is a bit of a hack. All of these things affect client-observed latency since obviously the client can't proceed until it gets the last packet. On mobile this is pretty important because mobile networks have massive packet loss. Anyway, that's mobile. In the datacenter all of these timers are way too high, by several orders of magnitude. 1ms is an outrageously long RTO in a datacenter fabric but 200ms is the minimum RTO allowed by the TCP RFCs.

by romed 8y ago

ivoras 8y ago

First actually useful and technical answer I've seen in this thread. But... which use cases of HTTP need us timers? It's not like gmail or other bloated web apps will load faster on my phone because of QUIC?

romed 8y ago
The timers are in TCP not HTTP. TCP only ACKs every other packet. If the stream pauses after an odd number of packets, the sender has to just wait for the receiver to perform a delayed ACK, for which there is a timer on both sides. The sender infers that the last packet has been lost if the ACK doesn't arrive after a while. The sender has a thing called the retransmission timeout (RTO) and the receiver's delayed ACK timeout (ATO) sets a lower bound on the RTO. Modern TCP stacks have a thing called the tail loss probe that just sends a retransmit early (before the RTO) if the stream has stopped at an odd boundary. This is a bit of a hack. All of these things affect client-observed latency since obviously the client can't proceed until it gets the last packet. On mobile this is pretty important because mobile networks have massive packet loss. Anyway, that's mobile. In the datacenter all of these timers are way too high, by several orders of magnitude. 1ms is an outrageously long RTO in a datacenter fabric but 200ms is the minimum RTO allowed by the TCP RFCs.