4 ms·

This article appears to be written from a solid Linux networking background, but not from an ISP networking background. ISPs at scale do not use software route

This article appears to be written from a solid Linux networking background, but not from an ISP networking background. ISPs at scale do not use software routers. They use ASIC routers (Juniper/Arista/Cisco/etc.), for many reasons 1) features 2) capacity 3) reliability. ASIC routers are capable of handling 100-1000x the throughput of the most over-provisioned Linux server (and that may even be an understatement). ASIC routers can also route packets with latency between 750us (0.75ms) and 10us (0.01ms!), complemented by multi-second (>GB) packet buffers. QoS is rarely used at scale, if anything only on the access layer, because transit has become so cheap that ISPs have more bandwidth than they know what to do with. These days, if a link is congested, it's not cost saving, but instead poor network planning. QoS also has very limited benefits at >100G scale. With that said, I feel that this article is definitely missing the full picture.

davecb 2y ago
Hi, Dave here. I recommend using a small box running LibreQoS adjacent to the big router. Large-scale routers based on Application-Specific ICs do a wonderful job, but are hard to change. Having a transparent fix in an inexpensive device now is way better than waiting and hoping that the router vendor can update their ASICs (:-)) I emphasised your problem in a video about the article, at https://vimeo.com/1017926413 https://vimeo.com/1017926413
inemesitaffia 2y ago
some isps are small and buy transit above $10/mbps
- iscoelho 2y ago
  That is almost impossible pricing these days at any bandwidth level unless you are either a) renewing an existing contract without negotiation b) are negligently bad at negotiation and are being taken advantage of c) are self dealing / sabotaging. (*unless you're in Australia) Bandwidth is cheap.
  - inemesitaffia 2y ago
    The world is a big place. I remember a country in Africa selling at $35/Mbps.
    - inemesitaffia 2y ago
      From a 2023 article A report by the Internet Society (ISOC) about IXPN and Kenyan IXP revealed that in early 2020, the port charge at IXPN was US$0.428 per month per Mbps (for a 1 Gbps port), while the cost of international IP transit is US$27.45 per Mbps per month (also for 1 Gbps capacity). Let me find more recent estimates. Just note that we aren't all in Ashburn, Frankfurt or Amsterdam
      - inemesitaffia 2y ago
        "Richard Obita, the Director, Planning, Research and Development at NITA-U, assured the MPs of his group’s aim to reduce the cost of internet further from the current US$70 for each megabit per second to US$35, which, if correct, is still astronomical compared to many countries." From Uganda April 2023. And Zimbabwe and Eritrea are more expensive

davecb 2y ago

Hi, Dave here. I recommend using a small box running LibreQoS adjacent to the big router. Large-scale routers based on Application-Specific ICs do a wonderful job, but are hard to change. Having a transparent fix in an inexpensive device now is way better than waiting and hoping that the router vendor can update their ASICs (:-)) I emphasised your problem in a video about the article, at https://vimeo.com/1017926413 https://vimeo.com/1017926413

inemesitaffia 2y ago

some isps are small and buy transit above $10/mbps

iscoelho 2y ago
That is almost impossible pricing these days at any bandwidth level unless you are either a) renewing an existing contract without negotiation b) are negligently bad at negotiation and are being taken advantage of c) are self dealing / sabotaging. (*unless you're in Australia) Bandwidth is cheap.
- inemesitaffia 2y ago
  The world is a big place. I remember a country in Africa selling at $35/Mbps.
  - inemesitaffia 2y ago
    From a 2023 article A report by the Internet Society (ISOC) about IXPN and Kenyan IXP revealed that in early 2020, the port charge at IXPN was US$0.428 per month per Mbps (for a 1 Gbps port), while the cost of international IP transit is US$27.45 per Mbps per month (also for 1 Gbps capacity). Let me find more recent estimates. Just note that we aren't all in Ashburn, Frankfurt or Amsterdam
    - inemesitaffia 2y ago
      "Richard Obita, the Director, Planning, Research and Development at NITA-U, assured the MPs of his group’s aim to reduce the cost of internet further from the current US$70 for each megabit per second to US$35, which, if correct, is still astronomical compared to many countries." From Uganda April 2023. And Zimbabwe and Eritrea are more expensive