4 ms·
This is a really good article. I do not know how to reach the author as there is one thing that bothers me about his default config. He set CAKE rtt to 30ms whi
by dtaht 3y ago
This is a really good article. I do not know how to reach the author as there is one thing that bothers me about his default config. He set CAKE rtt to 30ms which is OK, so long as 98% or so of the sites he wants to reach are within 30ms. Otherwise the default of 100ms is scaled to be able to manage bandwidth efficiently to sites around the world, and has been shown to work pretty well to real RTTs of 280ms. In general I have been recommending the rtt be set lower with the world consisting of CDNs, but not to 30ms.
Delighted to see vyos pick it up!
Next is the shaping vs variable rate issue, on outbound and inbound. On outbound - Ideally the fq_codel or cake algorithms are located right on the bottleneck link, and respond to changes in the available bandwidth from the upstream dynamically (asserted by ethernet pause frames for example), and managed underneath by BQL (ethernet) or AQL (some wifi(ath10k,ath11k, mt76, mt79)) , though true native fq_codel support exists for the ath9k) which store up enough data only to service one interrupt. These algorithms are much (20x!) lighter weight than shaping to a fixed rate is, and like I said
Codel was designed to deal with variable rate links. A fixed shaping rate was not what we intended, but has become one of the most common and finicky use cases, dang it. It is the FQ that matters most in most cases, however.
On inbound shaping, it's a SWAG, where we recommend 85% or so of the provided rate as a starting point. You might need more, you might need less, due to the behaviors of "slow start" in particular, it is never going to be accurate enough, and we keep encouraging the ISPs merely to manage their own egress well enough (ideally dynamically and without a shaper) so inbound shaping is not needed. Libreqos/preseem/bequant/paraqum all make middleboxes that do ISP level shaping. Libreqos (for whom I work these days) has pushed CAKE to where an ISP can do 10k subscribers at 25Gbit at 50% of cpu on a cheap ($2k) Xeon or Ryzen box.
So you get something that works well on the rrul test... in the morning but not the evening. Folk then complain that their bandwidth from their provider is variable rate and shaping via their other box does not work all the time, and I go back to hoping more ISPs get told how fix it at, egress, truly right at their end.
There is another project which perhaps can be ported to vyos, called cake-autorate, which attempts to use active measurements to cope with variable rates.