4 ms·

I don't get this explanation. Over one tcp connection, these "unlimited" requests are interleaved, blocking each other for periods of time. With multiple tcp

I don't get this explanation. Over one tcp connection, these "unlimited" requests are interleaved, blocking each other for periods of time. With multiple tcp connections I can have parallelism. It seems like for larger requests, you would suffer performance loss with http2? What am I missing here. Must be something.

cogman10 5y ago
Well, let's start from the bottom. For most computers and systems, there is only a signal network interface (and if not, there's usually just a single route from A to B.) So, when you say "I can have parallelism" you have to realize that even without a multiplexed TCP connection, the operating system is playing the same game. It is interleaving and blocking bits to the NIC even with multiple TCP connections. The parallelism you see with multiple TCP connections comes from the fact that the NIC is not (generally) saturated sending and receiving bits. So with that out of the way, where's the benefits? One is eliminating the TCP slow start [1]. TCP was designed generally with the notion that you'd only ever have 1 connection between 2 computers. As a result, it has baked into the protocol network congestion sensing protocols. More connections == more congestion control == slower requests. It also, however, means that the first request of every connection will be penalized as the connection warms up. A single connection pays that penalty once. Another is the latency problem. In order for a connection to be used, you need to send the data from the client to the server and then receive a response from the server. During that time, your connection is bound up (ignore http pipelining, it never worked well). You cannot add more bits down that connection, you have to wait for the server to process and send back response. If that round trip time takes 200ms, that's 200ms of sitting around doing nothing. During that time, over a 100Mbps connection your client could have sent 2.5MB worth of requests to the server. Because http request are typically very small (usually in the bytes), that's a lot of potential requests. This leads to the servers more than likely being able to saturate the connection which is what you want if you are trying to maximize performance. > It seems like for larger requests, you would suffer performance loss with http2? A little tricky, it depends on what you are measuring. A large request will block other requests going through while it is streaming. In that scenario, you'd still be saturating the network line but the perceived concurrency would be down. So, from a requests/sec standpoint, that could end up being lower. But from a data-throughput standpoint, you'll be better off still than http1 due to the better network congestion handling of a single TCP connection. (The ole latency v throughput problem). This is where we enter HTTP3, which bucks TCP as the primary request carrier for UDP messages instead. HTTP3 allows for much lower connection establishment costs (so making multiple QUIC connections isn't as expensive) but also contains protocol changes for errors which keeps an bad packet from blocking all the inbound packets (TCP will freeze the stream while it goes back up the connection to get the correct packet). [1] https://blog.stackpath.com/tcp-slow-start/ https://blog.stackpath.com/tcp-slow-start/
- theelous3 5y ago
  Very reasonable. I had forgotten to consider that this parallelism was smoke and mirrors from the ground up. This is an excellent comment and very helpful. Thank you.

cogman10 5y ago

Well, let's start from the bottom. For most computers and systems, there is only a signal network interface (and if not, there's usually just a single route from A to B.) So, when you say "I can have parallelism" you have to realize that even without a multiplexed TCP connection, the operating system is playing the same game. It is interleaving and blocking bits to the NIC even with multiple TCP connections. The parallelism you see with multiple TCP connections comes from the fact that the NIC is not (generally) saturated sending and receiving bits. So with that out of the way, where's the benefits? One is eliminating the TCP slow start [1]. TCP was designed generally with the notion that you'd only ever have 1 connection between 2 computers. As a result, it has baked into the protocol network congestion sensing protocols. More connections == more congestion control == slower requests. It also, however, means that the first request of every connection will be penalized as the connection warms up. A single connection pays that penalty once. Another is the latency problem. In order for a connection to be used, you need to send the data from the client to the server and then receive a response from the server. During that time, your connection is bound up (ignore http pipelining, it never worked well). You cannot add more bits down that connection, you have to wait for the server to process and send back response. If that round trip time takes 200ms, that's 200ms of sitting around doing nothing. During that time, over a 100Mbps connection your client could have sent 2.5MB worth of requests to the server. Because http request are typically very small (usually in the bytes), that's a lot of potential requests. This leads to the servers more than likely being able to saturate the connection which is what you want if you are trying to maximize performance. > It seems like for larger requests, you would suffer performance loss with http2? A little tricky, it depends on what you are measuring. A large request will block other requests going through while it is streaming. In that scenario, you'd still be saturating the network line but the perceived concurrency would be down. So, from a requests/sec standpoint, that could end up being lower. But from a data-throughput standpoint, you'll be better off still than http1 due to the better network congestion handling of a single TCP connection. (The ole latency v throughput problem). This is where we enter HTTP3, which bucks TCP as the primary request carrier for UDP messages instead. HTTP3 allows for much lower connection establishment costs (so making multiple QUIC connections isn't as expensive) but also contains protocol changes for errors which keeps an bad packet from blocking all the inbound packets (TCP will freeze the stream while it goes back up the connection to get the correct packet). [1] https://blog.stackpath.com/tcp-slow-start/ https://blog.stackpath.com/tcp-slow-start/

theelous3 5y ago
Very reasonable. I had forgotten to consider that this parallelism was smoke and mirrors from the ground up. This is an excellent comment and very helpful. Thank you.