5 ms·

To do NAT, you need to map (external) port numbers to (internal) IP addresses. This is done using connection tracking: tracking the state of the connection and

To do NAT, you need to map (external) port numbers to (internal) IP addresses. This is done using connection tracking: tracking the state of the connection and the appropriate mapping. And connection tracking gets expensive at scale.

londons_explore 4y ago
Not if you only allow each user 100 connections. That's 1200 bytes of ram per customer paying 100$ a month. And you can charge them an extra $10 per month for 'pro' internet and let them have 1000 connections for 'all the family'.
- stingraycharles 4y ago
  How do you ensure each user is capped at 100 connections without that check incurring additional resources?
  - londons_explore 4y ago
    You have a per user counter. So instead of 1200 bytes it's 1201 bytes per user.
    - stingraycharles 4y ago
      Memory isn’t the only dimension we care about. You’ve basically proposed an absolutely horrible solution, for both the end-user and the ISP. Something tells me you haven’t actually done any actual low level network engineering, and just brush all this off as “how hard can it be”.
    - kazen44 4y ago
      how are you going to keep this counter? Do you identify the bytes that are processed in individual flows? Which system will keep track of this? the control plane of the router maybe? great... you just added additional complexity instead of just pushing packets through a forwarding plane.
      - Dylan16807 4y ago
        When an unrecognized flow shows up, punt it to software. Handle the counter there, and if it overflows then you drop the packets. No need to add anything to the control plane.
        kazen44 4y ago
        "punting it to software" from a router with seperate control and forwarding planes perspective, is forwarding it to a control plane, instead of relying on the logic programmed inside the ASIC to forward traffic.
        Dylan16807 4y ago
        Sorry, I meant no need to add anything to the forwarding plane, or interfere with its efficiency at all. The point is, the really fast part doesn't need to be more complex. The part that handles new connections needs to be marginally more complicated, but not enough that it should really matter.
- shepherdjerred 4y ago
  Please don’t give Comcast ideas
- welterde 4y ago
  That's a laughable low limit. Even the "pro" plan would be marginal for a single person without running into limits from time to time. And nevermind power users that might do something with p2p or have a couple more devices connected to the network. But that's besides the point. Your home router can easily have millions of connections open (if they didn't skimp on the ram anyway), but if you have CGNAT boxes that do the same for tens of thousands of customers you also have to take into account that they have to move a lot of traffic. This means routing and doing NAT in software won't cut it anymore, but you need dedicated hardware coupled with very fast specialized memory to handle that traffic.
  - londons_explore 4y ago
    You can still do hardware NAT for the few thousand connections with the most packets and software NAT for everything else. I bet across even an ISP network of a million users, 80% of the traffic at any point in time is within 10,000 connections.
    - stingraycharles 4y ago
      You do realise that almost all connections are long-lived, and burst up and down in throughput? So the 10,000 “heaviest” connections right now are not the same as in, say, 3 seconds from now ? So you propose constantly swapping in and out connections from “hardware NAT” to “software NAT”? What heuristic will you use to decide which connections go where? Such a heuristic will probably look a lot like QoS, which is even more (much more!) resource hungry than NAT. At which point will the obvious conclusion be, “maybe the carriers who actually deal with these problems have a point, NAT is indeed a significant amount of complexity, and let’s be happy IPv6 starts to make actual economic sense?”
martinald 4y ago
Yes, I understand how NAT works. But CPU/memory is going down in cost way faster than bandwidth demand is increasing. Regardless, its way way cheaper than buying IPv4 blocks clearly, otherwise people wouldn't be doing it. Edit: ok, the problem isn't hardware, it's comedy license fees. https://itprice.com/juniper-price-list/cgn.html https://itprice.com/juniper-price-list/cgn.html $470k for a license to do CGNAT at 100gbit/sec. Surely these guys are opening themselves up to be replaced with some cheaper open source based software solution?
- stingraycharles 4y ago
  Software-defined networking is slowly becoming more popular, but it’s always going to be more resource intensive than these enterprise-grade routers that are typically implemented using FPGA / ASICs. Having said that, I’m often equally baffled at just how expensive modern networking hardware is, but as it’s pretty much all of these carrier grade networking solutions being this expensive, I’m assuming it’s somewhat justified. That doesn’t take away the fact that NAT just adds an expensive layer of complexity on top of it, and I can imagine that in the long term, IPv6 is starting to become much more attractive.
  - martinald 4y ago
    Probably a bit of a cartel for "enterprise grade" networking equipment, is my guess. Was similar in the late 90s/early 00s for web/database servers.
    - noizejoy 4y ago
      My (uneducated) guess would be to look at the way patents last too long. So society ends up suffering, rather than benefiting from IP protection.
  - Nextgrid 4y ago
    I’m not sure the price is justified, however the ISP market is extremely difficult/impossible to break through for startups or any company capable of building their own. It’s a self-fulfilling prophecy, the market is hard to break into (for other reasons besides networking equipment cost) so nobody who can actually do something about it is able to get in.

londons_explore 4y ago

Not if you only allow each user 100 connections. That's 1200 bytes of ram per customer paying 100$ a month. And you can charge them an extra $10 per month for 'pro' internet and let them have 1000 connections for 'all the family'.

stingraycharles 4y ago
How do you ensure each user is capped at 100 connections without that check incurring additional resources?
- londons_explore 4y ago
  You have a per user counter. So instead of 1200 bytes it's 1201 bytes per user.
  - stingraycharles 4y ago
    Memory isn’t the only dimension we care about. You’ve basically proposed an absolutely horrible solution, for both the end-user and the ISP. Something tells me you haven’t actually done any actual low level network engineering, and just brush all this off as “how hard can it be”.
  - kazen44 4y ago
    how are you going to keep this counter? Do you identify the bytes that are processed in individual flows? Which system will keep track of this? the control plane of the router maybe? great... you just added additional complexity instead of just pushing packets through a forwarding plane.
    - Dylan16807 4y ago
      When an unrecognized flow shows up, punt it to software. Handle the counter there, and if it overflows then you drop the packets. No need to add anything to the control plane.
      - kazen44 4y ago
        "punting it to software" from a router with seperate control and forwarding planes perspective, is forwarding it to a control plane, instead of relying on the logic programmed inside the ASIC to forward traffic.
        Dylan16807 4y ago
        Sorry, I meant no need to add anything to the forwarding plane, or interfere with its efficiency at all. The point is, the really fast part doesn't need to be more complex. The part that handles new connections needs to be marginally more complicated, but not enough that it should really matter.
shepherdjerred 4y ago
Please don’t give Comcast ideas
welterde 4y ago
That's a laughable low limit. Even the "pro" plan would be marginal for a single person without running into limits from time to time. And nevermind power users that might do something with p2p or have a couple more devices connected to the network. But that's besides the point. Your home router can easily have millions of connections open (if they didn't skimp on the ram anyway), but if you have CGNAT boxes that do the same for tens of thousands of customers you also have to take into account that they have to move a lot of traffic. This means routing and doing NAT in software won't cut it anymore, but you need dedicated hardware coupled with very fast specialized memory to handle that traffic.
- londons_explore 4y ago
  You can still do hardware NAT for the few thousand connections with the most packets and software NAT for everything else. I bet across even an ISP network of a million users, 80% of the traffic at any point in time is within 10,000 connections.
  - stingraycharles 4y ago
    You do realise that almost all connections are long-lived, and burst up and down in throughput? So the 10,000 “heaviest” connections right now are not the same as in, say, 3 seconds from now ? So you propose constantly swapping in and out connections from “hardware NAT” to “software NAT”? What heuristic will you use to decide which connections go where? Such a heuristic will probably look a lot like QoS, which is even more (much more!) resource hungry than NAT. At which point will the obvious conclusion be, “maybe the carriers who actually deal with these problems have a point, NAT is indeed a significant amount of complexity, and let’s be happy IPv6 starts to make actual economic sense?”

martinald 4y ago

Yes, I understand how NAT works. But CPU/memory is going down in cost way faster than bandwidth demand is increasing. Regardless, its way way cheaper than buying IPv4 blocks clearly, otherwise people wouldn't be doing it. Edit: ok, the problem isn't hardware, it's comedy license fees. https://itprice.com/juniper-price-list/cgn.html https://itprice.com/juniper-price-list/cgn.html $470k for a license to do CGNAT at 100gbit/sec. Surely these guys are opening themselves up to be replaced with some cheaper open source based software solution?

stingraycharles 4y ago
Software-defined networking is slowly becoming more popular, but it’s always going to be more resource intensive than these enterprise-grade routers that are typically implemented using FPGA / ASICs. Having said that, I’m often equally baffled at just how expensive modern networking hardware is, but as it’s pretty much all of these carrier grade networking solutions being this expensive, I’m assuming it’s somewhat justified. That doesn’t take away the fact that NAT just adds an expensive layer of complexity on top of it, and I can imagine that in the long term, IPv6 is starting to become much more attractive.
- martinald 4y ago
  Probably a bit of a cartel for "enterprise grade" networking equipment, is my guess. Was similar in the late 90s/early 00s for web/database servers.
  - noizejoy 4y ago
    My (uneducated) guess would be to look at the way patents last too long. So society ends up suffering, rather than benefiting from IP protection.
- Nextgrid 4y ago
  I’m not sure the price is justified, however the ISP market is extremely difficult/impossible to break through for startups or any company capable of building their own. It’s a self-fulfilling prophecy, the market is hard to break into (for other reasons besides networking equipment cost) so nobody who can actually do something about it is able to get in.