13 ms·

Bottleneck Bandwidth and RTT

246 points by 0123456 10y ago

otoburb 10y ago
Many TCP optimization algorithms report their performance improvements using CUBIC as their baseline. Will be very interesting to see how TCP optimization vendors adapt to the new Bottleneck Bandwidth & RTT patch. From an industry viewpoint, I wonder how this will perform over traditionally higher-latency and higher-loss wireless networks. As an aside, I love how small the patch is, weighing in at 875 LOC including comments.
Animats 10y ago
This is useful mostly for long-lived connections. As Google moves away from the many-short-connections HTTP model to the persistent connections of HTTP/2, connections live long enough that both bandwidth and delay can be discovered. This is much better than trying to estimate bandwidth from packet loss.
- contingencies 10y ago
  For those that don't know, someone revealed recently that Animats is John Nagle, who designed https://en.wikipedia.org/wiki/Nagle%27s_algorithm https://en.wikipedia.org/wiki/Nagle%27s_algorithm ... so this perhaps one of the most informed possible comments.
  - kyrra 10y ago
    It's on his profile comment here on HN. :)
    - hinkley 10y ago
      John Nagle revealed recently that Animats is John Nagle.
- AdamJacobMuller 10y ago
  Any idea of how to define "long lived" ? Excluding the HTTP/2 situations, obviously if you're fetching a single small resource (image or something) that takes <1s then that's short, but, where's the line there? Is something >1m long?
  - Animats 10y ago
    10 seconds in this code. static u32 bbr_min_rtt_win_sec = 10; /* min RTT filter window (in sec) */ The code also uses a moving average over 10 round trips. So that's what the filter needs to get a stable estimate. A point in the paper [1] is that this method is said to work well for maintained TCP connections with idle periods. That means HTTP/2 in practice. It would be interesting to see test data on this for large numbers of real connections. How much do bandwidth and delay vary in practice across ISP links, cable headends, and cellular links? [1] http://caia.swin.edu.au/cv/dahayes/content/networking2011-cdg-preprint.pdf http://caia.swin.edu.au/cv/dahayes/content/networking2011-cd...
    - apenwarr 10y ago
      The averaging period says more about how fast they expect min_rtt to change than how long it takes to calculate it. If the rtt is fairly stable, that measurement could be accurate much sooner than 10s. Super short lived sessions can really only go faster with tricks like increasing the initcwnd. Anything longer than that, I'd expect bbr to work well.
  - colechristensen 10y ago
    HTTP <2 doesn't bring up and tear down TCP connections for every asset. keep-alive is the default for 1.1 and supported in 1.0
  - yyyuuu 10y ago
    LET us twain walk aside from the rest; Now we are together privately, do you discard ceremony, Come! vouchsafe to me what has yet been vouchsafed to none—Tell me the whole story, Tell me what you would not tell your brother, wife, husband, or physician.
- watermoose 10y ago
  > This is much better than trying to estimate bandwidth from packet loss. This definitely seems like an improvement, however is it possible that this changed could result in one or more additional attack vectors? In addition, what about the additional resources needed to pull this off; how many fewer persistent connections could be maintained by a single server with the same specs?
  - apenwarr 10y ago
    There shouldn't be much overhead per stream. The estimators are just a few integers each, which is tiny compared to the TCP socket buffer itself. There could be added cpu overhead from implementing pacing if it isn't done carefully, but my understanding is the Linux pacing is done "right" so that it actually has very little or even negative overhead compared to traditional burst transmissions. (Also, it's generally safe to assume that Google cares more about server scalability than almost anyone else, so they wouldn't use this if it weren't pretty good on that front.)
_RPM 10y ago
This is really interesting work. I wish I was smart enough to do this kind of stuff.
- jeff_marshall 10y ago
  There is nothing magic about networking! If you are interested, start reading things and playing around. The issue, like most complex subjects, is that most people aren't interested enough in the details to get to the point where they are confident enough to do something innovative and present it to the world.
  - unmole 10y ago
    Do you have any pointers for a good place to start reading?
    - aidenn0 10y ago
      Comer's book "Computer Networks And Internets" is a decent introduction. If you are on a budget get one edition out-of-date; it's a college textbook so the previous edition is usually about 90% cheaper than the current edition. Once you feel comfortable with the basics, start going through RFCs for what you are interested in. In most cases the RFCs describe not just the protocol, but the reasoning behind it as well. For me though, I need to do more than read; I need to be "hands on." One example: When I was in college, I wanted to learn the IRC protocol better, and I noticed it was text based, so (after reading through a couple of RFCs) I connected to an IRC server with telnet in one window and the spec in another window. about 6 hours later I was finally able to connect and send messages. Those 6 hours were both less boring and more educational than 6 more hours of reading would be. Net results on my grades was either slightly negative, or a wash; I probably missed 2 or 3 classes during those 6 hours, but I got nearly double the next highest score on my Networking midterm 3 semesters later.
    - thr0waway1239 10y ago
      I once did some research on networking and related stuff - it was not my primary topic, but I remember a few things. During my reading, I found one of the best (as in readable) books was Doug Comer's "Internetworking With TCP/IP vol. 1" - an excellent theoretical reference. [1] However, skip the other volumes from Doug Comer (I think there are 3 volumes). For writing practical applications, Richard Stevens' "Unix Network Programming" [2] is usually recommended, I didn't find it an easy read though. Perhaps others can pitch in. For both the books suggested, getting a used old copy for cheap is a good idea because the core information was already there even in the first editions. Finally, read up on PlanetLab [3]. Its a fascinating project - a small scale internet built on top of a subset of nodes contributed by universities and research organizations across the globe - that people can contribute to, and if you actually manage to get into the developer's list and make a contribution, you can quite honestly claim to have pushed the state of the art forward. And lastly, be prepared to spend a good amount of time - I don't think it will be a fast or easy process. For whatever reason, I have found that the community around this work to be a little small, especially in comparison to how much it permeates pretty much everyone's life. [1] https://www.amazon.com/Internetworking-TCP-Vol-1-Principles-Architecture/dp/0130183806/ https://www.amazon.com/Internetworking-TCP-Vol-1-Principles-... [2] https://www.amazon.com/Unix-Network-Programming-Sockets-Networking/dp/0131411551/ https://www.amazon.com/Unix-Network-Programming-Sockets-Netw... [3] http://svn.planet-lab.org/ http://svn.planet-lab.org/
stephen_g 10y ago
This is very exciting, and I can't wait to see some performance data from it. Bufferbloat is a huge problem so it's awesome to see work being done in this area. It's really cool also that it can improve things just by patching servers! How does this interact sending traffic through routers using algorithms like fq_codel to reduce bufferbloat? Is it better to just have one or the other or do they work well together?
- bcook 10y ago
  CoDel is meant to solve bufferbloat at a single network node, not along the entire path. TCP BBR, as a consequence of more accurate congestion avoidance, seems to (hopefully) reduce bufferbloat along the entire path. Both of these algorithms should work together well since they do not really compete. (assuming that I understand...)
jnordwick 10y ago
Intra network, TCP slow start is often turned off to minimize latency on links especially ones that you have to respond very quickly to data initially or that is bursty. Google BBR seems to used the same exponential probing that slow start does, so I wonder how it will perform when you are staying in network and don't often have to worry about packet loss or congestion and want the link to start off at full throttle. Once BBR enters its steady state it intentionally cycles faster and slower, but this seems like it is creating additional latency when you don't want it. Think of a traffic burst that happens just as the link decides to cycle slower. It also seems like the protocol intentionally runs slower that possible as to not create buffer pressure on the receiving side, if I'm understanding this quick description properly: "then cruising at the estimated bandwidth to utilize the pipe without creating excess queue". The this line just scares me: "Occasionally, on an as-needed basis, it sends significantly slower to probe for RTT (PROBE_RTT mode). Google is going to make patches that work for them, but that doesn't always mean it will work for everybody else. This seems very close tailed to Google's traffic issues and serving HTTP over persistent connections, and not a general purpose feature, think of games, intranetwork low-latency applications, etc.
- honkhonkpants 10y ago
  "staying in network and don't often have to worry about packet loss or congestion." I don't think you would find many Googlers making this equivalence. If your network is highly utilized then it is characterized by both loss and congestion. The trick is optimizing throughput on a congested network without suffering from collapse. Whenever you see anyone talking about "traffic engineering" or "software-defined networking" you should just mentally substitute "packet loss". The only thing that an SDN management plane can do is instruct nodes on the network to drop. Then you can imagine that a highly utilized SDN-managed network has high loss. Edit: In Google's "B4" paper they show that their network operates at packet loss levels approaching 10%. See Figure 14 in http://cseweb.ucsd.edu/~vahdat/papers/b4-sigcomm13.pdf http://cseweb.ucsd.edu/~vahdat/papers/b4-sigcomm13.pdf
  - jnordwick 10y ago
    In my work, I don't deal with Internet traffic too much, so internal packet loss is mostly a sign of poor provisioning or technology (if it is the network's fault) or poor programming (if it is the endpoint's fault). Both of those are fixable. It is very rare to have true congestion and sometimes impossible (if it is a dedicated subnet with only a couple hosts). For us latency is more important that throughput, so our needs don't match Google's. There are a lot of cases like this.
kev009 10y ago
This is the same Van Jacobson who was instrumental in working through TCP congestion collapse growing pains over 30 years ago https://en.wikipedia.org/wiki/Network_congestion#Congestive_collapse https://en.wikipedia.org/wiki/Network_congestion#Congestive_...
quietplatypus 10y ago
The name is quite misleading for those not familiar with previous work. How's this different from TCP Vegas and FAST TCP which also use delay to infer the bottleneck bandwidth?
- apenwarr 10y ago
  bbr does not infer bandwidth using delay; it just measures the amount of data asked between two points in time (ie. it directly measures the bandwidth). It also directly measures the rtt. The big insight in bbr is that these two values are all you need (and you convert them into a "pace" and a window size); unlike eg. Vegas, you don't slow down just because latency increases, you slow down when measured bandwidth decreases. This makes it far more resilient, especially to other competing TCP flows.
nieksand 10y ago
Has anybody managed to find a preprint of the ACM queue paper floating around?
mhandley 10y ago
The linked article doesn't provide enough information to understand how they're using Acks to probe bottleneck bandwidth, but they've prior work on this. If it's similar to Van Jacobson's pathchar, I would have thought there might be "interesting" interactions with NICs that do TCP TSO and GRO (using the NIC to do TCP segmentation at the sender and packet aggregation at the receiver), as these will mess with timing and, in particular, how many acks get generated. Still, the authors are very well known and respected in the Networking community, so I expect they will have some way to handle this.
- apenwarr 10y ago
  It doesn't need to be as complicated as pathchar, because of a lucky fact: if you measure the end to end bandwidth, you are always measuring the bottleneck bandwidth :) So you can just count how many packets arrive at the other end between two time points and call that the bottleneck bandwidth.
- vsrinivas 10y ago
  You mean LRO instead of GRO, right?
wscott 10y ago
Very interesting, but I suspect like many of these it works best if you are only competing with only connections using the same approach. Which is probably why Google is talking about using this inside their backbone. This estimates the queues in the network and tries to keep them empty. TCP ramps until packets start dropping, so it spoils things for the BRR connection. Perhaps combined with Codel on to drop TCP packets early the two could play nicely together. Hmm, reading the code it says it does play well with TCP, but "requires the fq ("Fair Queue") pacing packet scheduler." In fact, later it says it MUST be used with fq. Hmm. BTW the code is very readable and well commented.
- apenwarr 10y ago
  I think the need for fq is just because that's how Linux does pacing, and pacing is essential for the accuracy of measurements used by bbr.
falcolas 10y ago
Is anyone able to speak to the compute and memory overhead this requires, in comparison with the loss-based algorithm? I ask on behalf of firewalls and layer 4 routers everywhere. Can this really just be patched in, with no changes to specalized hardware?
- Filligree 10y ago
  It's an endpoint-side algorithm. This goes on whatever terminates the TCP connection; servers, PCs, etc. Firewalls shouldn't be affected, and switches won't be. On behalf of everyone forced to use L4 switches, though, can you please reduce the buffer sizes? :P
apenwarr 10y ago
My favourite feature of this algorithm is it ought to fix the problems of very fast TCP over very long distances when there is nonzero packet loss. I think :) Traditionally recovering from packet loss can take several RTT, and if round trips are long, it might never get up to full speed. In theory, I think this will fix it.
dedalus 10y ago
The key point here is the deployment model: only change sender, no network or receiver side change. This means if you see even 5% improvement people are going to deploy it. Also the key architect seems Neal Cardwell and simply using Van Jacobson as a signoff/vetting authority. Eric Dumazet has long done a lot of work on TCP pacing (look at his prev patches) Basically all this means is that we have a form of TCP Pacing that can work (but suffers from classic prisoner's dilemma)
- tarrga 10y ago
  I work at Google in Bandwidth-SRE; watched this as it developed. Van's name is not on this patch as an appeal to authority --- he contributed meaningfully to the design. The analysis to support unloaded vs. loaded probes (to measure time-separated variables) was entirely his.

246 points by 0123456 10y ago

Many TCP optimization algorithms report their performance improvements using CUBIC as their baseline. Will be very interesting to see how TCP optimization vendors adapt to the new Bottleneck Bandwidth & RTT patch. From an industry viewpoint, I wonder how this will perform over traditionally higher-latency and higher-loss wireless networks. As an aside, I love how small the patch is, weighing in at 875 LOC including comments.

Animats 10y ago

This is useful mostly for long-lived connections. As Google moves away from the many-short-connections HTTP model to the persistent connections of HTTP/2, connections live long enough that both bandwidth and delay can be discovered. This is much better than trying to estimate bandwidth from packet loss.

contingencies 10y ago
For those that don't know, someone revealed recently that Animats is John Nagle, who designed https://en.wikipedia.org/wiki/Nagle%27s_algorithm https://en.wikipedia.org/wiki/Nagle%27s_algorithm ... so this perhaps one of the most informed possible comments.
- kyrra 10y ago
  It's on his profile comment here on HN. :)
  - hinkley 10y ago
    John Nagle revealed recently that Animats is John Nagle.
AdamJacobMuller 10y ago
Any idea of how to define "long lived" ? Excluding the HTTP/2 situations, obviously if you're fetching a single small resource (image or something) that takes <1s then that's short, but, where's the line there? Is something >1m long?
- Animats 10y ago
  10 seconds in this code. static u32 bbr_min_rtt_win_sec = 10; /* min RTT filter window (in sec) */ The code also uses a moving average over 10 round trips. So that's what the filter needs to get a stable estimate. A point in the paper [1] is that this method is said to work well for maintained TCP connections with idle periods. That means HTTP/2 in practice. It would be interesting to see test data on this for large numbers of real connections. How much do bandwidth and delay vary in practice across ISP links, cable headends, and cellular links? [1] http://caia.swin.edu.au/cv/dahayes/content/networking2011-cdg-preprint.pdf http://caia.swin.edu.au/cv/dahayes/content/networking2011-cd...
  - apenwarr 10y ago
    The averaging period says more about how fast they expect min_rtt to change than how long it takes to calculate it. If the rtt is fairly stable, that measurement could be accurate much sooner than 10s. Super short lived sessions can really only go faster with tricks like increasing the initcwnd. Anything longer than that, I'd expect bbr to work well.
- colechristensen 10y ago
  HTTP <2 doesn't bring up and tear down TCP connections for every asset. keep-alive is the default for 1.1 and supported in 1.0
- yyyuuu 10y ago
  LET us twain walk aside from the rest; Now we are together privately, do you discard ceremony, Come! vouchsafe to me what has yet been vouchsafed to none—Tell me the whole story, Tell me what you would not tell your brother, wife, husband, or physician.
watermoose 10y ago
> This is much better than trying to estimate bandwidth from packet loss. This definitely seems like an improvement, however is it possible that this changed could result in one or more additional attack vectors? In addition, what about the additional resources needed to pull this off; how many fewer persistent connections could be maintained by a single server with the same specs?
- apenwarr 10y ago
  There shouldn't be much overhead per stream. The estimators are just a few integers each, which is tiny compared to the TCP socket buffer itself. There could be added cpu overhead from implementing pacing if it isn't done carefully, but my understanding is the Linux pacing is done "right" so that it actually has very little or even negative overhead compared to traditional burst transmissions. (Also, it's generally safe to assume that Google cares more about server scalability than almost anyone else, so they wouldn't use this if it weren't pretty good on that front.)

_RPM 10y ago

This is really interesting work. I wish I was smart enough to do this kind of stuff.

jeff_marshall 10y ago
There is nothing magic about networking! If you are interested, start reading things and playing around. The issue, like most complex subjects, is that most people aren't interested enough in the details to get to the point where they are confident enough to do something innovative and present it to the world.
- unmole 10y ago
  Do you have any pointers for a good place to start reading?
  - aidenn0 10y ago
    Comer's book "Computer Networks And Internets" is a decent introduction. If you are on a budget get one edition out-of-date; it's a college textbook so the previous edition is usually about 90% cheaper than the current edition. Once you feel comfortable with the basics, start going through RFCs for what you are interested in. In most cases the RFCs describe not just the protocol, but the reasoning behind it as well. For me though, I need to do more than read; I need to be "hands on." One example: When I was in college, I wanted to learn the IRC protocol better, and I noticed it was text based, so (after reading through a couple of RFCs) I connected to an IRC server with telnet in one window and the spec in another window. about 6 hours later I was finally able to connect and send messages. Those 6 hours were both less boring and more educational than 6 more hours of reading would be. Net results on my grades was either slightly negative, or a wash; I probably missed 2 or 3 classes during those 6 hours, but I got nearly double the next highest score on my Networking midterm 3 semesters later.
  - thr0waway1239 10y ago
    I once did some research on networking and related stuff - it was not my primary topic, but I remember a few things. During my reading, I found one of the best (as in readable) books was Doug Comer's "Internetworking With TCP/IP vol. 1" - an excellent theoretical reference. [1] However, skip the other volumes from Doug Comer (I think there are 3 volumes). For writing practical applications, Richard Stevens' "Unix Network Programming" [2] is usually recommended, I didn't find it an easy read though. Perhaps others can pitch in. For both the books suggested, getting a used old copy for cheap is a good idea because the core information was already there even in the first editions. Finally, read up on PlanetLab [3]. Its a fascinating project - a small scale internet built on top of a subset of nodes contributed by universities and research organizations across the globe - that people can contribute to, and if you actually manage to get into the developer's list and make a contribution, you can quite honestly claim to have pushed the state of the art forward. And lastly, be prepared to spend a good amount of time - I don't think it will be a fast or easy process. For whatever reason, I have found that the community around this work to be a little small, especially in comparison to how much it permeates pretty much everyone's life. [1] https://www.amazon.com/Internetworking-TCP-Vol-1-Principles-Architecture/dp/0130183806/ https://www.amazon.com/Internetworking-TCP-Vol-1-Principles-... [2] https://www.amazon.com/Unix-Network-Programming-Sockets-Networking/dp/0131411551/ https://www.amazon.com/Unix-Network-Programming-Sockets-Netw... [3] http://svn.planet-lab.org/ http://svn.planet-lab.org/

stephen_g 10y ago

This is very exciting, and I can't wait to see some performance data from it. Bufferbloat is a huge problem so it's awesome to see work being done in this area. It's really cool also that it can improve things just by patching servers! How does this interact sending traffic through routers using algorithms like fq_codel to reduce bufferbloat? Is it better to just have one or the other or do they work well together?

bcook 10y ago
CoDel is meant to solve bufferbloat at a single network node, not along the entire path. TCP BBR, as a consequence of more accurate congestion avoidance, seems to (hopefully) reduce bufferbloat along the entire path. Both of these algorithms should work together well since they do not really compete. (assuming that I understand...)

jnordwick 10y ago

Intra network, TCP slow start is often turned off to minimize latency on links especially ones that you have to respond very quickly to data initially or that is bursty. Google BBR seems to used the same exponential probing that slow start does, so I wonder how it will perform when you are staying in network and don't often have to worry about packet loss or congestion and want the link to start off at full throttle. Once BBR enters its steady state it intentionally cycles faster and slower, but this seems like it is creating additional latency when you don't want it. Think of a traffic burst that happens just as the link decides to cycle slower. It also seems like the protocol intentionally runs slower that possible as to not create buffer pressure on the receiving side, if I'm understanding this quick description properly: "then cruising at the estimated bandwidth to utilize the pipe without creating excess queue". The this line just scares me: "Occasionally, on an as-needed basis, it sends significantly slower to probe for RTT (PROBE_RTT mode). Google is going to make patches that work for them, but that doesn't always mean it will work for everybody else. This seems very close tailed to Google's traffic issues and serving HTTP over persistent connections, and not a general purpose feature, think of games, intranetwork low-latency applications, etc.

honkhonkpants 10y ago
"staying in network and don't often have to worry about packet loss or congestion." I don't think you would find many Googlers making this equivalence. If your network is highly utilized then it is characterized by both loss and congestion. The trick is optimizing throughput on a congested network without suffering from collapse. Whenever you see anyone talking about "traffic engineering" or "software-defined networking" you should just mentally substitute "packet loss". The only thing that an SDN management plane can do is instruct nodes on the network to drop. Then you can imagine that a highly utilized SDN-managed network has high loss. Edit: In Google's "B4" paper they show that their network operates at packet loss levels approaching 10%. See Figure 14 in http://cseweb.ucsd.edu/~vahdat/papers/b4-sigcomm13.pdf http://cseweb.ucsd.edu/~vahdat/papers/b4-sigcomm13.pdf
- jnordwick 10y ago
  In my work, I don't deal with Internet traffic too much, so internal packet loss is mostly a sign of poor provisioning or technology (if it is the network's fault) or poor programming (if it is the endpoint's fault). Both of those are fixable. It is very rare to have true congestion and sometimes impossible (if it is a dedicated subnet with only a couple hosts). For us latency is more important that throughput, so our needs don't match Google's. There are a lot of cases like this.

kev009 10y ago

This is the same Van Jacobson who was instrumental in working through TCP congestion collapse growing pains over 30 years ago https://en.wikipedia.org/wiki/Network_congestion#Congestive_collapse https://en.wikipedia.org/wiki/Network_congestion#Congestive_...

quietplatypus 10y ago

The name is quite misleading for those not familiar with previous work. How's this different from TCP Vegas and FAST TCP which also use delay to infer the bottleneck bandwidth?

apenwarr 10y ago
bbr does not infer bandwidth using delay; it just measures the amount of data asked between two points in time (ie. it directly measures the bandwidth). It also directly measures the rtt. The big insight in bbr is that these two values are all you need (and you convert them into a "pace" and a window size); unlike eg. Vegas, you don't slow down just because latency increases, you slow down when measured bandwidth decreases. This makes it far more resilient, especially to other competing TCP flows.

nieksand 10y ago

Has anybody managed to find a preprint of the ACM queue paper floating around?

mhandley 10y ago

The linked article doesn't provide enough information to understand how they're using Acks to probe bottleneck bandwidth, but they've prior work on this. If it's similar to Van Jacobson's pathchar, I would have thought there might be "interesting" interactions with NICs that do TCP TSO and GRO (using the NIC to do TCP segmentation at the sender and packet aggregation at the receiver), as these will mess with timing and, in particular, how many acks get generated. Still, the authors are very well known and respected in the Networking community, so I expect they will have some way to handle this.

apenwarr 10y ago
It doesn't need to be as complicated as pathchar, because of a lucky fact: if you measure the end to end bandwidth, you are always measuring the bottleneck bandwidth :) So you can just count how many packets arrive at the other end between two time points and call that the bottleneck bandwidth.
vsrinivas 10y ago
You mean LRO instead of GRO, right?

wscott 10y ago

Very interesting, but I suspect like many of these it works best if you are only competing with only connections using the same approach. Which is probably why Google is talking about using this inside their backbone. This estimates the queues in the network and tries to keep them empty. TCP ramps until packets start dropping, so it spoils things for the BRR connection. Perhaps combined with Codel on to drop TCP packets early the two could play nicely together. Hmm, reading the code it says it does play well with TCP, but "requires the fq ("Fair Queue") pacing packet scheduler." In fact, later it says it MUST be used with fq. Hmm. BTW the code is very readable and well commented.

apenwarr 10y ago
I think the need for fq is just because that's how Linux does pacing, and pacing is essential for the accuracy of measurements used by bbr.

falcolas 10y ago

Is anyone able to speak to the compute and memory overhead this requires, in comparison with the loss-based algorithm? I ask on behalf of firewalls and layer 4 routers everywhere. Can this really just be patched in, with no changes to specalized hardware?

Filligree 10y ago
It's an endpoint-side algorithm. This goes on whatever terminates the TCP connection; servers, PCs, etc. Firewalls shouldn't be affected, and switches won't be. On behalf of everyone forced to use L4 switches, though, can you please reduce the buffer sizes? :P

apenwarr 10y ago

My favourite feature of this algorithm is it ought to fix the problems of very fast TCP over very long distances when there is nonzero packet loss. I think :) Traditionally recovering from packet loss can take several RTT, and if round trips are long, it might never get up to full speed. In theory, I think this will fix it.

dedalus 10y ago

The key point here is the deployment model: only change sender, no network or receiver side change. This means if you see even 5% improvement people are going to deploy it. Also the key architect seems Neal Cardwell and simply using Van Jacobson as a signoff/vetting authority. Eric Dumazet has long done a lot of work on TCP pacing (look at his prev patches) Basically all this means is that we have a form of TCP Pacing that can work (but suffers from classic prisoner's dilemma)

tarrga 10y ago
I work at Google in Bandwidth-SRE; watched this as it developed. Van's name is not on this patch as an appeal to authority --- he contributed meaningfully to the design. The analysis to support unloaded vs. loaded probes (to measure time-separated variables) was entirely his.