4 ms·

the wiki article is fine. however it too misses some context. the internet has no flow control - any sender can issue as many packets as their egress link will

the wiki article is fine. however it too misses some context. the internet has no flow control - any sender can issue as many packets as their egress link will bear. Since the subsequent links are over-provisioned, if the sum of the input rates exceeds the output rate, some packets may need to be dropped. good or ill - the solution is to put a controller on the sender that examines the traffic statistics and tries to adapt its sending rate so as not to overflow the network capacity. since everyone is doing this, we also kind of want to make sure that everyone sharing an output link adapts to something close to a fraction of that bandwidth divided by the total number of TCP flows. this is fairness. without getting into the highly involved details you can already see that this is a really interesting control/game theory problem. I find it particularly weird, astounding even, that this situation is really quite ideal for a tragedy of the commons. except there is no tragedy. everyone can cheat, but almost no one does. sure - you'd have to recompile your kernel, but anyone could ship 'network turbocharger 2.0' that forces all the other TCPs on the network to back off and allocate as much intermediate bandwidth as you like. a very easy way to do this as Netscape discovered is to use multiple connections. the pieces are all the same size, but you can take as many as you like. obviously that situation degenerates very quickly. but it hasn't. and it leaves the designers of wide area services that don't just use one TCP connection quite at a loss about how much bandwidth they should take. as far as the survey goes, I can think of all sorts of reasons why the IETF would care. one reason is that uptake of new work is pretty hard - and has killed lots of great protocol work. I'm pretty surprised that BBR has such rapid uptake - the technical situation remains a little unclear. for me the interesting part is the discussion of the measurement technique

Sesse__ 6y ago
> everyone can cheat, but almost no one does Try running a site with big files for downloads. Every week or so, you'll be visited by someone who runs 200+ simultaneous connections for the same file, keeps opening more, and ignores HTTP errors (like “too busy”). There are also the “IW wars”, where CDNs compete by increasing their initial window more and more to gain an edge. This will be exponentially worse with QUIC, when tuning these knobs becomes more accessible to non-experts.
- Animats 6y ago
  everyone can cheat, but almost no one does That's the point of "fair queuing". The idea is that if you send too much, you just compete with your own packets, and thus are not rewarded for sending too much. This discourages hogging the bandwidth. But fair queuing is computationally expensive, and is rarely seen in core routers, where some FPGA is doing routing. It's also not seen enough in home routers, where fast LAN hits slower uplink and a FIFO queue gets long, increasing latency. That's "bufferbloat". Quite fixable. Now get AT&T and Comcast to cooperate. From the paper, "Classic congestion control questions on fairness guarantees and convergence need to be re-answered for the new rate-based congestion control paradigm." Yes. They're looking at performance for individual nodes. This will be exponentially worse with QUIC Or when Google uses it to improve their responsiveness vs. others?
  - Sesse__ 6y ago
    The point is that QUIC (and related efforts) is a library, not implemented in the kernel. This makes it way easier to deploy, but also way easier (in practice) to change in bad ways.
- youngtaff 6y ago
  CDNs also adapt the initcwnd based on the destination network the packets are being sent too
vitus 6y ago
> I'm pretty surprised that BBR has such rapid uptake - the technical situation remains a little unclear. As I mentioned in a sibling comment, BBR has high uptake among Google properties, but not necessarily much outside; many of the other big players are still using CUBIC, since it's tried and tested. Raw counts aren't a super useful metric, especially if many sibling domains (e.g. google.com, google.com.hk) can count toward that same number. That said, BBR is available as an option in the mainline Linux kernel, and I wouldn't be surprised to see at least a bit of cargo culting explaining its popularity ("it works for Google, so..."). (Alexa top 250 sites are no longer free to view, apparently, but the top 50 contain at least four google properties. Other website rankings: https://moz.com/top500 https://moz.com/top500 shows no fewer than 45 hits for "google", not to mention other properties such as Youtube or Blogger; https://trends.netcraft.com/topsites https://trends.netcraft.com/topsites has 33 hits for "Google LLC" out of their top 100.) I'm personally surprised by the widespread use of algorithms outside of Cubic, personally. Some of these are obviously in-house, e.g. Akamai CC, and there are some cases of, say, CTCP from Windows Server 2008 machines, but Cubic is the default on Linux, newer versions of Windows, and even macOS, which I assume covers a large majority of servers.
- the8472 6y ago
  > As I mentioned in a sibling comment, BBR has high uptake among Google properties, but not necessarily much outside As you said, it's available in linux behind a simple sysctl and here's a cloudflare blog post describing just that[0]. And netflix contributed the equivalent to freebsd[1]. And here's a blog post about AWS cloudfront seeing improvements from it[2]. TFA also mentions Steam. So it's more than just google. [0] https://blog.cloudflare.com/http-2-prioritization-with-nginx/ https://blog.cloudflare.com/http-2-prioritization-with-nginx... [1] https://reviews.freebsd.org/D21582 https://reviews.freebsd.org/D21582 [2] https://aws.amazon.com/blogs/networking-and-content-delivery/tcp-bbr-congestion-control-with-amazon-cloudfront/ https://aws.amazon.com/blogs/networking-and-content-delivery...

Sesse__ 6y ago

> everyone can cheat, but almost no one does Try running a site with big files for downloads. Every week or so, you'll be visited by someone who runs 200+ simultaneous connections for the same file, keeps opening more, and ignores HTTP errors (like “too busy”). There are also the “IW wars”, where CDNs compete by increasing their initial window more and more to gain an edge. This will be exponentially worse with QUIC, when tuning these knobs becomes more accessible to non-experts.

Animats 6y ago
everyone can cheat, but almost no one does That's the point of "fair queuing". The idea is that if you send too much, you just compete with your own packets, and thus are not rewarded for sending too much. This discourages hogging the bandwidth. But fair queuing is computationally expensive, and is rarely seen in core routers, where some FPGA is doing routing. It's also not seen enough in home routers, where fast LAN hits slower uplink and a FIFO queue gets long, increasing latency. That's "bufferbloat". Quite fixable. Now get AT&T and Comcast to cooperate. From the paper, "Classic congestion control questions on fairness guarantees and convergence need to be re-answered for the new rate-based congestion control paradigm." Yes. They're looking at performance for individual nodes. This will be exponentially worse with QUIC Or when Google uses it to improve their responsiveness vs. others?
- Sesse__ 6y ago
  The point is that QUIC (and related efforts) is a library, not implemented in the kernel. This makes it way easier to deploy, but also way easier (in practice) to change in bad ways.
youngtaff 6y ago
CDNs also adapt the initcwnd based on the destination network the packets are being sent too

vitus 6y ago

> I'm pretty surprised that BBR has such rapid uptake - the technical situation remains a little unclear. As I mentioned in a sibling comment, BBR has high uptake among Google properties, but not necessarily much outside; many of the other big players are still using CUBIC, since it's tried and tested. Raw counts aren't a super useful metric, especially if many sibling domains (e.g. google.com, google.com.hk) can count toward that same number. That said, BBR is available as an option in the mainline Linux kernel, and I wouldn't be surprised to see at least a bit of cargo culting explaining its popularity ("it works for Google, so..."). (Alexa top 250 sites are no longer free to view, apparently, but the top 50 contain at least four google properties. Other website rankings: https://moz.com/top500 https://moz.com/top500 shows no fewer than 45 hits for "google", not to mention other properties such as Youtube or Blogger; https://trends.netcraft.com/topsites https://trends.netcraft.com/topsites has 33 hits for "Google LLC" out of their top 100.) I'm personally surprised by the widespread use of algorithms outside of Cubic, personally. Some of these are obviously in-house, e.g. Akamai CC, and there are some cases of, say, CTCP from Windows Server 2008 machines, but Cubic is the default on Linux, newer versions of Windows, and even macOS, which I assume covers a large majority of servers.

the8472 6y ago
> As I mentioned in a sibling comment, BBR has high uptake among Google properties, but not necessarily much outside As you said, it's available in linux behind a simple sysctl and here's a cloudflare blog post describing just that[0]. And netflix contributed the equivalent to freebsd[1]. And here's a blog post about AWS cloudfront seeing improvements from it[2]. TFA also mentions Steam. So it's more than just google. [0] https://blog.cloudflare.com/http-2-prioritization-with-nginx/ https://blog.cloudflare.com/http-2-prioritization-with-nginx... [1] https://reviews.freebsd.org/D21582 https://reviews.freebsd.org/D21582 [2] https://aws.amazon.com/blogs/networking-and-content-delivery/tcp-bbr-congestion-control-with-amazon-cloudfront/ https://aws.amazon.com/blogs/networking-and-content-delivery...