4 ms·

Because running multiple TCP connections in parallel plays havoc with TCP congestion control and also plays poorly with the TCP slow-start logic. Every TCP conn

by Lukasa 12y ago

Because running multiple TCP connections in parallel plays havoc with TCP congestion control and also plays poorly with the TCP slow-start logic. Every TCP connection begins its receive window again and so it starts small, so fetching many moderately-sized or large resources (think images) will cost you many round trips you didn't need to spend.

bwross 12y ago
Additionally, a TCP connection is essentially an operating system resource; you need to set aside a port and space for a send and receive buffer. It might seem fine for a client to open hundreds connections, but imagine being a server with thousands of clients all opening hundreds of connections to you. You very quickly run out of resources and either have to close connections or reject new connections.
- otterley 12y ago
  That hasn't been a practical problem in many years. Most servers have gigabytes of RAM and a 64-bit kernel nowadays.
  - getsat 12y ago
    Linux starts to act weird around 200,000 concurrent connections in my experience, even with aggressive sysctl tuning. You end up with weird edge cases like netstat literally taking 15 minutes of CPU time (in kernel) before it dumps the list of connections to stdout. Not sure about FreeBSD or any other OSes.
    - btmorex 12y ago
      Not saying netstat isn't slow, but: # time sh -c 'netstat -tn | wc -l' 486206 real 0m13.538s user 0m1.698s sys 0m10.380s It still works with a whole lost of connections. (in fairness, only about 130k were connected)
      - getsat 12y ago
        The problem is that it doesn't scale linearly. There's some O(n4) algorithm being used in netstat or some kernel syscalls or something. Once you go over 250k, things get _really_ weird.
        patrickmcmanus 12y ago
        try ss -nt instead.. it uses netlink sockets instead of /proc and generally scales much better
        getsat 12y ago
        Thanks for the tip!
bsdetector 12y ago
If you start out making 6 TCP connections then in a perfect network their receive windows all expand in parallel -- 6 times faster than a single HTTP2 connection. You're likely to see several HTTP2 connection made as well. There'll at least be a second one in case the first SYN is lost, just like with HTTP 1. The key TCP benefit is keeping a connection open. That can be done with keep-alive as well.
jude- 12y ago
Moreover, it's bad from a QoS standpoint. Opening many TCP streams in parallel is not fair to other users sharing the routers between you and the server. You'd get more than your fair share of bandwidth.
- otterley 12y ago
  Only if you don't assume that everyone else will also use the same number of streams in parallel. If everyone else's utilization is increased by the same factor, the utilization balance should remain the same.
  - jude- 12y ago
    The goodput decreases for everyone then, since each flow requires a 3-RTT handshake and 1-2-RTT tear-down. This is particularly bad if you're starting up a lot of small flows, where the control-plane information becomes a non-negligible fraction of the total data sent. I think ideally, we'd create a TCP variant where localhost maintains a per-destination receiving window for all flows to that destination, so flows running in parallel or flows started in rapid succession won't have to start their windows at 0 and slowly increase them. Moreover, this way congestion control applies to all packet flows for a (source, destination) pair, instead of to individual flows. HTTP/2 and HTTP pipelining take a crack at this by running multiple application-level flows (i.e. HTTP requests) through the same receiving window (i.e. the same TCP socket), but they're not the only application-level protocols that could stand to benefit.
- viraptor 12y ago
  Doesn't that depend entirely on the queuing and session mapping algorithm? If you profile for bandwidth-per-source-ip, shouldn't that apply exactly the same restriction regardless of how many connections are started? (with multiple-connections people losing a bit because connection start/stop takes bandwidth they could use for data instead)
  - jude- 12y ago
    Yes and no. Yes in theory, using bandwidth-per-source-IP caps like you suggest could be used to solve this. No in practice, because the prevalence of NATs puts many users behind the same IP address, meaning that unfair users can still hog the upstream bandwidth from everyone else in the same NAT. What we really want is something more like bandwidth-per-end-host caps.
otterley 12y ago
I don't buy the congestion-control argument. Transit routers have millions of connections going through them at any given point. What's an extra 10x in concurrency to them?
- patrickmcmanus 12y ago
  Its a fair question but the h2 approach is doing the right thing on balance. The issue isn't really computational or memory requirements. The startup phase of a TCP stream is essentially not governed by congestion control feedback because there hasn't been enough (or maybe any) feedback yet. It is initially controlled by a constant (IW) and then slowly feels its way before dynamically finding the right rate. IW can generally range from 2 to 10 segments. Whether this is too much or too little for any individual circumstance is somewhat immaterial - its generally going to be wrong just because that's the essence of the probing phase - you start with a guess and go from there. Each stream has a beginning a middle and an end. 1 large stream has 1 beginning 1 (large) middle and 1 end, but N small streams have N beginnings N (smaller) middles, and N ends. The amount of data in the beginning is not a factor of the stream size (other than it being the max), it is rather governed by the latency and bandwidth of the network. So more streams means more data gets carried in the startup phase. If the beginning is known to be a poor performing stage (and for TCP it is) then creating more of them and having them cover more of the data is a bad strategy. In practice, IW is too small for the median stream - but there is a wide distribution of "right sizes" so its ridiculously hard to get right.. maybe it is IW=10 and the right size is 30 segments; but that's one stream 3x too small- it isn't 20x or 50x too small, so when you open 50 parallel tcp connections you are effectively sending at IW * 50. And that does indeed cause congestion and packet loss.. and its not the kind of "I dropped 1 packet from a run of 25 please use fast-retransmit or SACK to fix it for me" packet loss we like to see.. its more of the "that was a train wreck I need slow timers on the order of hundreds of milliseconds to try again for me" packet loss that brings tears to my eyes. One of the reasons for this goes back to the N beginnings problem - if you lose a SYN or your last data packet the recovery process is inherently much slower, and N streams have N times more SYNS and "last" packets than 1 stream does. Oh, and 50 isn't an exaggeration. HTTP routinely wants to generate a burst of 100 simultaneous requests these days (which is why header compression when doing multiplexing is critical - but that's another post). So the busier larger flow both induces less loss and is more responsive when it experiences loss. That's a win. And after all that you still have the priority problem. 50 uncoordinated streams all entering the network at slightly different times with slightly different amounts of data will be extremely chaotic wrt which data gets transferred first. And "first" matters a lot to the web - things like js/css/fonts all block you from using the web, but images might not.. and even within those images some are more important than others (some might not even turn out to be on the screen at first - oh wait, you just scrolled I need to change those priorities). Providing a coordination mechanism for that is one of the real pieces of untapped potential hiding in h2's approach to mux everything together. There is a downside. If you have a single non-induced loss (i.e. due to some other data source) and it impacts the early part of the 1 single tcp connection then it impacts all the other virtual streams because of tcp's in-order delivery properties. If they were split into N tcp connections then only one of them would be impacted. This is a much discussed property in the networking community, and I've seen it in the wild - but nobody has demonstrated that it is a significant operational problem. The h2 arrangement is the right thing to do in a straightforward TLS/HTTP1-COMPATIBLE-SEMANTIC/TCP.. making further improvements will involve breaking out of that traditional tcp box a bit (quic is an example, minion is also related, even mosh is related) and is appropriately separated as next-stage work that wasn't part of h2. Its considerably more experimental.

by Lukasa 12y ago

bwross 12y ago

Additionally, a TCP connection is essentially an operating system resource; you need to set aside a port and space for a send and receive buffer. It might seem fine for a client to open hundreds connections, but imagine being a server with thousands of clients all opening hundreds of connections to you. You very quickly run out of resources and either have to close connections or reject new connections.

otterley 12y ago
That hasn't been a practical problem in many years. Most servers have gigabytes of RAM and a 64-bit kernel nowadays.
- getsat 12y ago
  Linux starts to act weird around 200,000 concurrent connections in my experience, even with aggressive sysctl tuning. You end up with weird edge cases like netstat literally taking 15 minutes of CPU time (in kernel) before it dumps the list of connections to stdout. Not sure about FreeBSD or any other OSes.
  - btmorex 12y ago
    Not saying netstat isn't slow, but: # time sh -c 'netstat -tn | wc -l' 486206 real 0m13.538s user 0m1.698s sys 0m10.380s It still works with a whole lost of connections. (in fairness, only about 130k were connected)
    - getsat 12y ago
      The problem is that it doesn't scale linearly. There's some O(n4) algorithm being used in netstat or some kernel syscalls or something. Once you go over 250k, things get _really_ weird.
      - patrickmcmanus 12y ago
        try ss -nt instead.. it uses netlink sockets instead of /proc and generally scales much better
        getsat 12y ago
        Thanks for the tip!

bsdetector 12y ago

If you start out making 6 TCP connections then in a perfect network their receive windows all expand in parallel -- 6 times faster than a single HTTP2 connection. You're likely to see several HTTP2 connection made as well. There'll at least be a second one in case the first SYN is lost, just like with HTTP 1. The key TCP benefit is keeping a connection open. That can be done with keep-alive as well.

jude- 12y ago

Moreover, it's bad from a QoS standpoint. Opening many TCP streams in parallel is not fair to other users sharing the routers between you and the server. You'd get more than your fair share of bandwidth.

otterley 12y ago
Only if you don't assume that everyone else will also use the same number of streams in parallel. If everyone else's utilization is increased by the same factor, the utilization balance should remain the same.
- jude- 12y ago
  The goodput decreases for everyone then, since each flow requires a 3-RTT handshake and 1-2-RTT tear-down. This is particularly bad if you're starting up a lot of small flows, where the control-plane information becomes a non-negligible fraction of the total data sent. I think ideally, we'd create a TCP variant where localhost maintains a per-destination receiving window for all flows to that destination, so flows running in parallel or flows started in rapid succession won't have to start their windows at 0 and slowly increase them. Moreover, this way congestion control applies to all packet flows for a (source, destination) pair, instead of to individual flows. HTTP/2 and HTTP pipelining take a crack at this by running multiple application-level flows (i.e. HTTP requests) through the same receiving window (i.e. the same TCP socket), but they're not the only application-level protocols that could stand to benefit.
viraptor 12y ago
Doesn't that depend entirely on the queuing and session mapping algorithm? If you profile for bandwidth-per-source-ip, shouldn't that apply exactly the same restriction regardless of how many connections are started? (with multiple-connections people losing a bit because connection start/stop takes bandwidth they could use for data instead)
- jude- 12y ago
  Yes and no. Yes in theory, using bandwidth-per-source-IP caps like you suggest could be used to solve this. No in practice, because the prevalence of NATs puts many users behind the same IP address, meaning that unfair users can still hog the upstream bandwidth from everyone else in the same NAT. What we really want is something more like bandwidth-per-end-host caps.

otterley 12y ago

I don't buy the congestion-control argument. Transit routers have millions of connections going through them at any given point. What's an extra 10x in concurrency to them?

patrickmcmanus 12y ago
Its a fair question but the h2 approach is doing the right thing on balance. The issue isn't really computational or memory requirements. The startup phase of a TCP stream is essentially not governed by congestion control feedback because there hasn't been enough (or maybe any) feedback yet. It is initially controlled by a constant (IW) and then slowly feels its way before dynamically finding the right rate. IW can generally range from 2 to 10 segments. Whether this is too much or too little for any individual circumstance is somewhat immaterial - its generally going to be wrong just because that's the essence of the probing phase - you start with a guess and go from there. Each stream has a beginning a middle and an end. 1 large stream has 1 beginning 1 (large) middle and 1 end, but N small streams have N beginnings N (smaller) middles, and N ends. The amount of data in the beginning is not a factor of the stream size (other than it being the max), it is rather governed by the latency and bandwidth of the network. So more streams means more data gets carried in the startup phase. If the beginning is known to be a poor performing stage (and for TCP it is) then creating more of them and having them cover more of the data is a bad strategy. In practice, IW is too small for the median stream - but there is a wide distribution of "right sizes" so its ridiculously hard to get right.. maybe it is IW=10 and the right size is 30 segments; but that's one stream 3x too small- it isn't 20x or 50x too small, so when you open 50 parallel tcp connections you are effectively sending at IW * 50. And that does indeed cause congestion and packet loss.. and its not the kind of "I dropped 1 packet from a run of 25 please use fast-retransmit or SACK to fix it for me" packet loss we like to see.. its more of the "that was a train wreck I need slow timers on the order of hundreds of milliseconds to try again for me" packet loss that brings tears to my eyes. One of the reasons for this goes back to the N beginnings problem - if you lose a SYN or your last data packet the recovery process is inherently much slower, and N streams have N times more SYNS and "last" packets than 1 stream does. Oh, and 50 isn't an exaggeration. HTTP routinely wants to generate a burst of 100 simultaneous requests these days (which is why header compression when doing multiplexing is critical - but that's another post). So the busier larger flow both induces less loss and is more responsive when it experiences loss. That's a win. And after all that you still have the priority problem. 50 uncoordinated streams all entering the network at slightly different times with slightly different amounts of data will be extremely chaotic wrt which data gets transferred first. And "first" matters a lot to the web - things like js/css/fonts all block you from using the web, but images might not.. and even within those images some are more important than others (some might not even turn out to be on the screen at first - oh wait, you just scrolled I need to change those priorities). Providing a coordination mechanism for that is one of the real pieces of untapped potential hiding in h2's approach to mux everything together. There is a downside. If you have a single non-induced loss (i.e. due to some other data source) and it impacts the early part of the 1 single tcp connection then it impacts all the other virtual streams because of tcp's in-order delivery properties. If they were split into N tcp connections then only one of them would be impacted. This is a much discussed property in the networking community, and I've seen it in the wild - but nobody has demonstrated that it is a significant operational problem. The h2 arrangement is the right thing to do in a straightforward TLS/HTTP1-COMPATIBLE-SEMANTIC/TCP.. making further improvements will involve breaking out of that traditional tcp box a bit (quic is an example, minion is also related, even mosh is related) and is appropriately separated as next-stage work that wasn't part of h2. Its considerably more experimental.