4 ms·

The biggest problem with CAKE/CoDel is that if you're not deploying it on the bottleneck device, you have to artificially limit the download/upload speed ("band

by p1mrx 3y ago

The biggest problem with CAKE/CoDel is that if you're not deploying it on the bottleneck device, you have to artificially limit the download/upload speed ("bandwidth 280mbit" "bandwidth 45mbit" in this example) in order to create a new bottleneck. Finding the correct limits is nontrivial because they might change over time. Anecdotally, switching from DOCSIS 3.0 to 3.1 seems to have improved my worst-case latency, because DOCSIS 3.1 includes a CAKE-like algorithm (PIE) in the modem itself: https://www.cablelabs.com/blog/how-docsis-3-1-reduces-latency-with-active-queue-management https://www.cablelabs.com/blog/how-docsis-3-1-reduces-latenc...

yencabulator 3y ago
> Finding the correct limits is nontrivial because they might change over time. It's even worse than that, a lot of consumer Internet connections are bottlenecked by some shared link, which means your max bandwidth literally depends on what the other customers are doing at the moment. There is no "correct limit".
davecb 3y ago
The bottleneck device can change with every different destination, so CAKE and fq_codel find it the same way older TCP code did: increase the load until they get a timeout or a "slow down, you idiot" flag (:-)) The initial bandwidth setting is really to get it started at a good value for your usual bottleneck device, such as your local cable modem. Always starting off as if you had fast ethernet to the ISP when you actually have a super-el-cheapo link is a waste of time and effort. Even if you have a good algorithm like CAKE. --dave
- p1mrx 3y ago
  When I used CoDel on OpenWrt years ago, setting a 1 Mbps limit prevented the link from ever going faster. If it's now smart enough to discover the correct limit, then that's an interesting/useful development.
  - pixl97 3y ago
    I mean setting a static traffic control limit was generally a high water mark, and in general even if you had excess bandwidth you didn't want to exceed it. Now, to get around that some devices perform regular speed tests and dynamically adjust the high water mark. This said there are some limits at which you should perform these tests as too often and you may affect the actual applications you're trying to run.
  - p1mrx 3y ago
    Followup: I just tested OpenWrt 23.05.2 + luci-app-sqm + cake with https://speed.cloudflare.com/ https://speed.cloudflare.com/ With the limit set to 1 Mbps (1000/1000), my upload latency dropped from 80ms to 25ms, but speed was hard-limited to 1000/1000. With the limit rasied to 1G/1G, cake stopped working and my upload latency returned to 80ms. So I stand by my original comment. You still have to configure the speed limits manually.
    - wtallis 3y ago
      You're right: CoDel and derivatives like fq_codel and cake don't auto-tune anything on timescales much longer than the interval parameter which defaults to 100ms. And fq_codel doesn't even do traffic shaping. But I think davecb may have been confusing traffic shaper limits with TCP congestion control behaviors, and maybe the impact of a large TCP initial window releasing a sudden burst of packets that may be large enough to build up a bit of a queue on particularly slow links. (It was a serious problem in the ADSL era; now, only wireless gets that slow, and large bursts of packets are as likely to help as not when frame aggregation enters the picture.)
Animats 3y ago
> The biggest problem with CAKE/CoDel is that if you're not deploying it on the bottleneck device. Right. This is a scheme for trying to manage the flows from the end you control so that the clueless FIFO nodes out there don't overload. Flow management should be at the point where multiple flows feed into a bottleneck. Usually at the point where the LAN connects to the outside world, the telco modem or cable box. And, in the other direction, where bulk bandwidth goes into limited bandwidth to the consumer at the upstream end. I'm encouraged to hear that this finally got into DOCSIS. Now if only we could get it into AT&T modems.

by p1mrx 3y ago

yencabulator 3y ago

> Finding the correct limits is nontrivial because they might change over time. It's even worse than that, a lot of consumer Internet connections are bottlenecked by some shared link, which means your max bandwidth literally depends on what the other customers are doing at the moment. There is no "correct limit".

davecb 3y ago

The bottleneck device can change with every different destination, so CAKE and fq_codel find it the same way older TCP code did: increase the load until they get a timeout or a "slow down, you idiot" flag (:-)) The initial bandwidth setting is really to get it started at a good value for your usual bottleneck device, such as your local cable modem. Always starting off as if you had fast ethernet to the ISP when you actually have a super-el-cheapo link is a waste of time and effort. Even if you have a good algorithm like CAKE. --dave

p1mrx 3y ago
When I used CoDel on OpenWrt years ago, setting a 1 Mbps limit prevented the link from ever going faster. If it's now smart enough to discover the correct limit, then that's an interesting/useful development.
- pixl97 3y ago
  I mean setting a static traffic control limit was generally a high water mark, and in general even if you had excess bandwidth you didn't want to exceed it. Now, to get around that some devices perform regular speed tests and dynamically adjust the high water mark. This said there are some limits at which you should perform these tests as too often and you may affect the actual applications you're trying to run.
- p1mrx 3y ago
  Followup: I just tested OpenWrt 23.05.2 + luci-app-sqm + cake with https://speed.cloudflare.com/ https://speed.cloudflare.com/ With the limit set to 1 Mbps (1000/1000), my upload latency dropped from 80ms to 25ms, but speed was hard-limited to 1000/1000. With the limit rasied to 1G/1G, cake stopped working and my upload latency returned to 80ms. So I stand by my original comment. You still have to configure the speed limits manually.
  - wtallis 3y ago
    You're right: CoDel and derivatives like fq_codel and cake don't auto-tune anything on timescales much longer than the interval parameter which defaults to 100ms. And fq_codel doesn't even do traffic shaping. But I think davecb may have been confusing traffic shaper limits with TCP congestion control behaviors, and maybe the impact of a large TCP initial window releasing a sudden burst of packets that may be large enough to build up a bit of a queue on particularly slow links. (It was a serious problem in the ADSL era; now, only wireless gets that slow, and large bursts of packets are as likely to help as not when frame aggregation enters the picture.)

Animats 3y ago

> The biggest problem with CAKE/CoDel is that if you're not deploying it on the bottleneck device. Right. This is a scheme for trying to manage the flows from the end you control so that the clueless FIFO nodes out there don't overload. Flow management should be at the point where multiple flows feed into a bottleneck. Usually at the point where the LAN connects to the outside world, the telco modem or cable box. And, in the other direction, where bulk bandwidth goes into limited bandwidth to the consumer at the upstream end. I'm encouraged to hear that this finally got into DOCSIS. Now if only we could get it into AT&T modems.