6 ms·

Why are NATs getting costly? Don't quite understand.

kazen44 4y ago
because maintaining state for GCNAT tables is far more complex then just forwarding packets. routers doing NAT are thus more expensive then those just doing simple forwarding. Also, in some countries ISP's need to map the use of a specific ip address to a specific subscriber for law enforcment purposes. GCNAT is no exception to this and creates a large amount of overhead because the public IPV4 prefix space is shared between multiple customers.
- TedDoesntTalk 4y ago
  > because maintaining state for GCNAT tables is far more complex then just forwarding packets. But it’s a solved problem with mature solutions, decades old. Is it really financially expensive?
  - kazen44 4y ago
    compared to rolling out IPv6? definitely, especially on the longer term. For instance, most Core/Edge routers (my experience is mainly with juniper MX series, but i assume the model is roughly the same for other vendors), you need specific licenses or interface card's to do stateful services like NAT. Compared to doing IPv6, which is "just forwarding packets" and doesn't require the hardware to track state in nearly the same manner. Most serious core/edge hardware vendors also do not put IPv6 behind licenses compared to CGNAT and other NAT-like features, because packet based forwarding is the most basic functionality a router should provide. Routers which are able to do less state, also are frequently far less expensive.
    - kortilla 4y ago
      You’re presenting a false dichotomy. The choice for an ISP today is not “v4 or v6”, it’s either “v4 or v4+v6”. A v6 only connection in the US is unusable.
      - est31 4y ago
        The more traffic you can get onto ipv6 the less stress is on the v4 infrastructure. Each v6 connection is one your CGNAT doesn't have to provide an ipv4 port for.
        kortilla 4y ago
        So what? That’s still just a scaling factor at that point and still requires you to have v4 cgnat infrastructure + ipv6.
        est31 4y ago
        You don't have to beef up your v4 infra as much though. Think 4 powerful v4 routers instead of 5 or something. If the traffic to the big streaming providers doesn't have to run through these routers, you can save a lot. Same goes for the ipv4 address space you have to rent/buy. The more connections are on ipv6, the less public ipv4 addresses you need to have. So ipv6 support might be saving you costs already in a dual stack setting.
        kortilla 4y ago
        Take a step back to the wider context of a brand new ISP though. If you’re rushing to market like Starlink appears to be, you either implement just v4 and scale later or implement both v4/v6 up front. Until there is a bunch of exclusive v6 stuff customers will be up in arms over missing, the answer of which thing to prioritize is obvious.
        est31 4y ago
        Yeah I guess it's the same as with the inter satellite communication which is promised for later, but not implemented yet so that they get at least some product out to customers. I don't think dual stack is that hard to do for entirely new networks though. Also, one of the reasons to do satellite internet is lower latency which is a bit hurt by CGNAT infrastructure. Last, generally brand new ISPs are in the situation that they have a hard time of getting ipv4 address space. The incumbents, especially the older ones, were around when ipv4 addesses were still plenty so they usually have way less problems with ipv4 address space. Starlink only has 166k ipv4 addresses according to https://ipinfo.io/AS14593 https://ipinfo.io/AS14593 . Compare this to AT&T which has over a hundred million for their AS 7018 https://ipinfo.io/AS7018 https://ipinfo.io/AS7018 alone, and there are other AS numbers they have like AS20057 with 7 million ipv4s. This roughly matches the number of AT&T customers while Starlink has more than double the number of subscribers than its number of public IPs, with growth ahead.
  - wongarsu 4y ago
    The same could be said about IPv6. I think the point is that IPv6 scales better with traffic increases, to the point where switching from CGNAT to IPv6 becomes financially attractive.
  - swinglock 4y ago
    What do you mean, solved problem?
  - Taywee 4y ago
    A problem being solved doesn't mean the current solution is inexpensive or optimal.
stingraycharles 4y ago
To do NAT, you need to map (external) port numbers to (internal) IP addresses. This is done using connection tracking: tracking the state of the connection and the appropriate mapping. And connection tracking gets expensive at scale.
- londons_explore 4y ago
  Not if you only allow each user 100 connections. That's 1200 bytes of ram per customer paying 100$ a month. And you can charge them an extra $10 per month for 'pro' internet and let them have 1000 connections for 'all the family'.
  - stingraycharles 4y ago
    How do you ensure each user is capped at 100 connections without that check incurring additional resources?
    - londons_explore 4y ago
      You have a per user counter. So instead of 1200 bytes it's 1201 bytes per user.
      - stingraycharles 4y ago
        Memory isn’t the only dimension we care about. You’ve basically proposed an absolutely horrible solution, for both the end-user and the ISP. Something tells me you haven’t actually done any actual low level network engineering, and just brush all this off as “how hard can it be”.
      - kazen44 4y ago
        how are you going to keep this counter? Do you identify the bytes that are processed in individual flows? Which system will keep track of this? the control plane of the router maybe? great... you just added additional complexity instead of just pushing packets through a forwarding plane.

because maintaining state for GCNAT tables is far more complex then just forwarding packets. routers doing NAT are thus more expensive then those just doing simple forwarding. Also, in some countries ISP's need to map the use of a specific ip address to a specific subscriber for law enforcment purposes. GCNAT is no exception to this and creates a large amount of overhead because the public IPV4 prefix space is shared between multiple customers.

TedDoesntTalk 4y ago
> because maintaining state for GCNAT tables is far more complex then just forwarding packets. But it’s a solved problem with mature solutions, decades old. Is it really financially expensive?
- kazen44 4y ago
  compared to rolling out IPv6? definitely, especially on the longer term. For instance, most Core/Edge routers (my experience is mainly with juniper MX series, but i assume the model is roughly the same for other vendors), you need specific licenses or interface card's to do stateful services like NAT. Compared to doing IPv6, which is "just forwarding packets" and doesn't require the hardware to track state in nearly the same manner. Most serious core/edge hardware vendors also do not put IPv6 behind licenses compared to CGNAT and other NAT-like features, because packet based forwarding is the most basic functionality a router should provide. Routers which are able to do less state, also are frequently far less expensive.
  - kortilla 4y ago
    You’re presenting a false dichotomy. The choice for an ISP today is not “v4 or v6”, it’s either “v4 or v4+v6”. A v6 only connection in the US is unusable.
    - est31 4y ago
      The more traffic you can get onto ipv6 the less stress is on the v4 infrastructure. Each v6 connection is one your CGNAT doesn't have to provide an ipv4 port for.
      - kortilla 4y ago
        So what? That’s still just a scaling factor at that point and still requires you to have v4 cgnat infrastructure + ipv6.
        est31 4y ago
        You don't have to beef up your v4 infra as much though. Think 4 powerful v4 routers instead of 5 or something. If the traffic to the big streaming providers doesn't have to run through these routers, you can save a lot. Same goes for the ipv4 address space you have to rent/buy. The more connections are on ipv6, the less public ipv4 addresses you need to have. So ipv6 support might be saving you costs already in a dual stack setting.
        kortilla 4y ago
        Take a step back to the wider context of a brand new ISP though. If you’re rushing to market like Starlink appears to be, you either implement just v4 and scale later or implement both v4/v6 up front. Until there is a bunch of exclusive v6 stuff customers will be up in arms over missing, the answer of which thing to prioritize is obvious.
        est31 4y ago
        Yeah I guess it's the same as with the inter satellite communication which is promised for later, but not implemented yet so that they get at least some product out to customers. I don't think dual stack is that hard to do for entirely new networks though. Also, one of the reasons to do satellite internet is lower latency which is a bit hurt by CGNAT infrastructure. Last, generally brand new ISPs are in the situation that they have a hard time of getting ipv4 address space. The incumbents, especially the older ones, were around when ipv4 addesses were still plenty so they usually have way less problems with ipv4 address space. Starlink only has 166k ipv4 addresses according to https://ipinfo.io/AS14593 https://ipinfo.io/AS14593 . Compare this to AT&T which has over a hundred million for their AS 7018 https://ipinfo.io/AS7018 https://ipinfo.io/AS7018 alone, and there are other AS numbers they have like AS20057 with 7 million ipv4s. This roughly matches the number of AT&T customers while Starlink has more than double the number of subscribers than its number of public IPs, with growth ahead.
- wongarsu 4y ago
  The same could be said about IPv6. I think the point is that IPv6 scales better with traffic increases, to the point where switching from CGNAT to IPv6 becomes financially attractive.
- swinglock 4y ago
  What do you mean, solved problem?
- Taywee 4y ago
  A problem being solved doesn't mean the current solution is inexpensive or optimal.

stingraycharles 4y ago

To do NAT, you need to map (external) port numbers to (internal) IP addresses. This is done using connection tracking: tracking the state of the connection and the appropriate mapping. And connection tracking gets expensive at scale.

londons_explore 4y ago
Not if you only allow each user 100 connections. That's 1200 bytes of ram per customer paying 100$ a month. And you can charge them an extra $10 per month for 'pro' internet and let them have 1000 connections for 'all the family'.
- stingraycharles 4y ago
  How do you ensure each user is capped at 100 connections without that check incurring additional resources?
  - londons_explore 4y ago
    You have a per user counter. So instead of 1200 bytes it's 1201 bytes per user.
    - stingraycharles 4y ago
      Memory isn’t the only dimension we care about. You’ve basically proposed an absolutely horrible solution, for both the end-user and the ISP. Something tells me you haven’t actually done any actual low level network engineering, and just brush all this off as “how hard can it be”.
    - kazen44 4y ago
      how are you going to keep this counter? Do you identify the bytes that are processed in individual flows? Which system will keep track of this? the control plane of the router maybe? great... you just added additional complexity instead of just pushing packets through a forwarding plane.