4 ms·

The ISP said they got only a limited set of IPv4 addresses, those are assigned to their CGNAT gateways. Which are expensive, especially for an Indian Reservatio

The ISP said they got only a limited set of IPv4 addresses, those are assigned to their CGNAT gateways. Which are expensive, especially for an Indian Reservation, not the deep pockets of some MSP. The users are behind CG-NAT. But instead of using IPv6 which is cheaper (no need to maintain CG-NAT, translation devices, or deal with traffic that is being routed way more expensively) the Roku devices are only streaming over IPv4. Each new user that adds an IPv4 only device adds additional load the CG-NAT and additional capacity will need to be provisioned. That is an additional expense and burden. Most of my traffic at my house (on Comcast) is over IPv6, because most if not all streaming services now support IPv6 for content delivery, so the small amount of data that may need to go over IPv4 when the majority can go over IPv6 reduces the load on IPv4.

zamadatix 4y ago
Realistically they are already running a large scale NAT device anyways, otherwise the v6 only clients wouldn't be able to reach v4 only internet services (hello HN server), and if they had planned to CG-NAT from the start it probably could have all been done on the same pair of internet edge routers for much less than 300k additional expense.
- Symbiote 4y ago
  As the article says, the Roku traffic is the overwhelming majority of that ipv4 traffic. HN and similar mostly-text websites would barely show up on the statistics.
  - zamadatix 4y ago
    NAT scales in both dimensions, you're just as likely to need a larger box for a larger connection table as you are a larger bandwidth.
    - noAnswer 4y ago
      At some point you also need more IPv4-addresses. There are only 65535 ports to be mapped.
      - zamadatix 4y ago
        At some point but remember it's not "after 65535 connections per IP" it's "after 65535 connections of the same type to the same IP per IP". That is to saythese NATs where 156.78.92.154 is a single public address on the NATing box: 156.78.92.154:6781<->8.8.8.8:53 UDP 156.78.92.154:6781<->8.8.8.8:53 TCP 156.78.92.154:6781<->8.8.4.4:53 TCP 156.78.92.154:6781<->8.8.4.4:53 UDP Can all exist simultaneously even though only 1 port is in use and we haven't even started changing the destination IP or ports yet (e.g. 80 and 443 as the destination won't double count and thousands of web servers on different addresses can be accessed via the same source port). With CG-NAT you can usually support somewhere around 30k customers on the free /24 you get from ARIN for having only IPv6 and needing space to translate in (for new orgs like new ISPs this is assigned immediately out of a reserved block, bypassing the waitlist for existing orgs trying to get free reclaimed space).
    - ArchOversight 4y ago
      It's also about ports. If I have 10 Gbps of IPv4 flow but my devices are licensed for and operate on 1 Gbps, I need to license 10 1 Gbps ports. When I was last working at an ISP, various CG-NAT solutions charged not just based on the connection table, but also there was various licensing for various slide-in cards. If the ISP is having to buy additional hardware/additional ports on upstream providers just to power their CG-NAT that is an additional cost. If the POP where you have your CG-NAT doesn't have more bandwidth available you end up running your ports hot... so now you either need to rent a new location, get your connectivity up there and start figuring out how to route the traffic. It's not as simple, and it all costs copious amounts of money. Especially if your IPv6 can more easily be distributed across multiple POP's with multiple forms of connectivity and traffic shaped using BGP or other solutions thereby reducing the load on a single upstream port.
brokencode 4y ago
Why does the IPv4 address need to be publicly exposed outside of a customer’s LAN? I thought you could set up NAT on a router to translate local IPv4 addresses to something IPv6 that is exposed publicly. Is this simply bad planning from the ISP where they didn’t handle it correctly? Or is there something I’m not understanding about NAT? I think in an ideal world all devices would be using IPv6. But I thought it would be common knowledge among network engineers that many devices still use IPv4, so you have to either handle it somehow or tell your customers that some of their devices simply won’t work.
- zamadatix 4y ago
  CG-NAT (NATing centrally at the ISP's internet edge) is cheaper/simpler than something like 464xlat (NATing v4 locally over v6) since you can do the former on 2 boxes instead of 20,000. That said the 2nd option is much cooler :).
  - ArchOversight 4y ago
    464XLAT still requires IPv4 boxes on the edge to translate IPv6 traffic back to IPv4. Whether that is two boxes or 20,000, the same is true for other CG-NAT solutions. Someone somewhere is bearing the cost of translating.
    - zamadatix 4y ago
      Usually you need a special box at the consumer side to do 464XLAT, it's not something you can just ask your customer's Netgear to do and it's usually more expensive if you want to provide it as the customer's rented router. CG-NAT however looks completely normal to all gear (other than the particular numbers assigned) except the 2 edge boxes. It's the ultimate cost saving kludge.
      - cmeacham98 4y ago
        Most modern OSes (I know for a fact Android, iOS, and Windows) support automatically doing the 4->6 translation on their side (as a matter of fact, some cellular networks in the US are ipv6 only). I'm unsure if consumer routers would pass on the appropriate RA flag to tell the OS they need to do this in their default configuration however.

zamadatix 4y ago

Realistically they are already running a large scale NAT device anyways, otherwise the v6 only clients wouldn't be able to reach v4 only internet services (hello HN server), and if they had planned to CG-NAT from the start it probably could have all been done on the same pair of internet edge routers for much less than 300k additional expense.

Symbiote 4y ago
As the article says, the Roku traffic is the overwhelming majority of that ipv4 traffic. HN and similar mostly-text websites would barely show up on the statistics.
- zamadatix 4y ago
  NAT scales in both dimensions, you're just as likely to need a larger box for a larger connection table as you are a larger bandwidth.
  - noAnswer 4y ago
    At some point you also need more IPv4-addresses. There are only 65535 ports to be mapped.
    - zamadatix 4y ago
      At some point but remember it's not "after 65535 connections per IP" it's "after 65535 connections of the same type to the same IP per IP". That is to saythese NATs where 156.78.92.154 is a single public address on the NATing box: 156.78.92.154:6781<->8.8.8.8:53 UDP 156.78.92.154:6781<->8.8.8.8:53 TCP 156.78.92.154:6781<->8.8.4.4:53 TCP 156.78.92.154:6781<->8.8.4.4:53 UDP Can all exist simultaneously even though only 1 port is in use and we haven't even started changing the destination IP or ports yet (e.g. 80 and 443 as the destination won't double count and thousands of web servers on different addresses can be accessed via the same source port). With CG-NAT you can usually support somewhere around 30k customers on the free /24 you get from ARIN for having only IPv6 and needing space to translate in (for new orgs like new ISPs this is assigned immediately out of a reserved block, bypassing the waitlist for existing orgs trying to get free reclaimed space).
  - ArchOversight 4y ago
    It's also about ports. If I have 10 Gbps of IPv4 flow but my devices are licensed for and operate on 1 Gbps, I need to license 10 1 Gbps ports. When I was last working at an ISP, various CG-NAT solutions charged not just based on the connection table, but also there was various licensing for various slide-in cards. If the ISP is having to buy additional hardware/additional ports on upstream providers just to power their CG-NAT that is an additional cost. If the POP where you have your CG-NAT doesn't have more bandwidth available you end up running your ports hot... so now you either need to rent a new location, get your connectivity up there and start figuring out how to route the traffic. It's not as simple, and it all costs copious amounts of money. Especially if your IPv6 can more easily be distributed across multiple POP's with multiple forms of connectivity and traffic shaped using BGP or other solutions thereby reducing the load on a single upstream port.

brokencode 4y ago

Why does the IPv4 address need to be publicly exposed outside of a customer’s LAN? I thought you could set up NAT on a router to translate local IPv4 addresses to something IPv6 that is exposed publicly. Is this simply bad planning from the ISP where they didn’t handle it correctly? Or is there something I’m not understanding about NAT? I think in an ideal world all devices would be using IPv6. But I thought it would be common knowledge among network engineers that many devices still use IPv4, so you have to either handle it somehow or tell your customers that some of their devices simply won’t work.

zamadatix 4y ago
CG-NAT (NATing centrally at the ISP's internet edge) is cheaper/simpler than something like 464xlat (NATing v4 locally over v6) since you can do the former on 2 boxes instead of 20,000. That said the 2nd option is much cooler :).
- ArchOversight 4y ago
  464XLAT still requires IPv4 boxes on the edge to translate IPv6 traffic back to IPv4. Whether that is two boxes or 20,000, the same is true for other CG-NAT solutions. Someone somewhere is bearing the cost of translating.
  - zamadatix 4y ago
    Usually you need a special box at the consumer side to do 464XLAT, it's not something you can just ask your customer's Netgear to do and it's usually more expensive if you want to provide it as the customer's rented router. CG-NAT however looks completely normal to all gear (other than the particular numbers assigned) except the 2 edge boxes. It's the ultimate cost saving kludge.
    - cmeacham98 4y ago
      Most modern OSes (I know for a fact Android, iOS, and Windows) support automatically doing the 4->6 translation on their side (as a matter of fact, some cellular networks in the US are ipv6 only). I'm unsure if consumer routers would pass on the appropriate RA flag to tell the OS they need to do this in their default configuration however.