10 ms·

Leveraging Class E address space to mitigate IPv4 exhaustion issues in GKE

28 points by l1n 2y ago

deleted 2y ago
[deleted]
RadiozRadioz 2y ago
> However not all enterprises or applications are ready for IPv6 yet. I don't think this statement will ever be false in my lifetime.
ddtaylor 2y ago
Anything but IPv6 it would seem.
- deleted 2y ago
  [deleted]
- ianburrell 2y ago
  I wish that Kubernetes had started as IPv6-only or IPv6-first. IPv6 is perfect for giving each pod its own address. There is more space for giving each host a subnet, or giving each pod its own address in the subnet. That would have gotten rid of the CNI overlay networks. It would also made barrier between the internal and external networks. It would require running NAT64 and DNS64 in most clusters.
  - magicalhippo 2y ago
    > I wish that Kubernetes had started as IPv6-only or IPv6-first Seems a bit strange to me that they didn't, given how you'll want to use some sort of ingress anyway, so IPv4 could be delegated to those edge points.
- tialaramex 2y ago
  > "It is remarkable, love," he said, looking at Nell for the first time, "how much money you can make shovelling back the tide." [ "Diamond Age" by Neal Stephenson ]
jandrese 2y ago
> Snap initially considered migrating to IPv6, but concerns about application readiness and interoperability led them to adopt dual-stack GKE nodes and GKE pods (IPv6 + Class E IPv4). This solution mitigated IP exhaustion and provided Snap with multiple years of IP address scale needed to support future growth and reduce operational overhead. In addition, this approach also aligned with Snap’s long-term strategy for migrating to IPv6. Yet another horrible hack to avoid having to actually learn IPv6.
- dmitrygr 2y ago
  The first time someone calls you a horse, you punch him on the nose; the second time someone calls you a horse, you call him a jerk; but the third time someone calls you a horse, well then, perhaps it's time to go shopping for a saddle.
- yjftsjthsd-h 2y ago
  How does using both avoid learning v6?
  - jandrese 2y ago
    Years of experience with "dual stack" organizations that completely neglect the IPv6 side or do stuff like install MITM firewalls that can't handle IPv6 so the addresses are only useful internally.
- Hikikomori 2y ago
  Who says they don't want to learn? When your provider doesn't support IPv6-only for all your purposes (I don't know about Google but Azure and AWS doesn't do that) then you'll just exhaust RFC1918 and not work on a solution for it?
hypeatei 2y ago
Interesting read, but it seems like a waste of time to keep finding little tricks and band aids with IPv4 when v6 exists. > However not all enterprises or applications are ready for IPv6 yet Ah yeah, there it is. Please, just fucking prioritize upgrading to IPv6 and be done with it. Frustrating. Cloud providers need to hurry up as well, Azure still doesn't fully support v6 on their app services (web servers) either. It's in public preview but has been a roadmap item for longer. It also comes with certain caveats like what tiers can use it.
dangerlibrary 2y ago
It never made sense to me why the authors of the IPv6 standard chose to use an entirely different address format, instead of extending address space in a backwards-compatible way. Why not just append/prepend the 12 additional bytes onto existing IPv4 addresses, and write the standard such that all valid IPv4 addresses are also valid IPv6 addresses? I'm not an RFC author, but something like "All existing IPv4 addresses will be reachable under the 0.0.0.0.0.0.0.0.0.0.0.0 prefix in the IPv6 Standard" seems like it would've made migration relatively trivial. The draft standard is 26 years old. The official standard is 7 years old, and we are still reading articles about how "not all enterprises or applications are ready for IPv6 yet." This question is coming from a place of genuine confusion and curiosity - I really don't get it. Did the authors of the standard just assume that migration and adoption would be easier than they've turned out to be? Was it a fairness issue where somehow this would have granted dominion over huge swaths of the new address space to existing players?
- ianburrell 2y ago
  Because writing the addresses is not the important part. You can write IPv6 addresses as dotted decimal. The length is the problem, IPv6 format is more compact. Addresses are not the hard part in upgrading. The embedding doesn't help with compatibility. IPv4 still can't access IPv6. IPv6 can't access IPv4. It actually breaks NAT64, which depends on special prefix. Also, extending IPv4 address space has the problem that bake the misallocation into IPv6. IPv4 is broken in the small chunks that makes the routing table large. It also means that new organizations will have a hard time getting address space cause they need to get IPv4 addresses.
  - dangerlibrary 2y ago
    "The length is the problem, IPv6 format is more compact." "IPv4 still can't access IPv6. IPv6 can't access IPv4." I guess my point is that these statements represent choices made by humans. Leaving the decimal representation aside, I don't get why they made these choices. If every IPv4 address were a valid IPv6 address, then these statements wouldn't be true.
tgma 2y ago
djb on IPv6 incompatibility being a bad design choice: https://cr.yp.to/djbdns/ipv6mess.html https://cr.yp.to/djbdns/ipv6mess.html
- kstrauser 2y ago
  His points were wrong 22 years ago, and they continue to be wrong today. It's a poor document to cite as a rationale for avoiding IPv6.
  - tgma 2y ago
    Huh? It is not a rationale for avoiding IPv6. Quite the contrary. It is a critique on how IPv6 was designed without being interoperable with IPv4 address space. It is very clear that he was right given there are 22 more years of evidence of IPv4 still being very much with us, and foreseeable future.
    - ay 2y ago
      Quote from the doc: “ Answer: We go through every place that 4-byte IPv4 addresses appear, and allow 16-byte IPv6 addresses in the same place.” - this text is pretty much the definition of “IPv6 transition”.
      - throw0101d 2y ago
        Also from the document: > Unfortunately, instead of simply allowing 16-byte A records, people introduced new ``AAAA records'' into the DNS protocol, creating several unnecessary complications in DNS software. "Simply". Yeah. Until some non-upgraded client gets an 'extended-A' record response that's the "wrong" size (according itself) and drops the packet (which is the sane thing to do security-wise) and then you've just broken your DNS completely: IPv4 no longer works for some people, and neither does IPv4+. So you have to wait until all client software understands extended-A records—but what's the motivation for enabling new client software if no one is enabling server-side records because there's no client software. Chicken-egg. Which is one argument against IPv6: why enable client-side when there's no content, and why bother server-side if there are no client requests? Chicken-egg.
solatic 2y ago
Incredibly unsafe. If Class E is reserved "for future use", maybe that could include public use. The first rule of hardening production reliability is, make sure it doesn't just work, but that it will continue to work. Tomorrow, some customer of yours gets assigned addresses from Class E. Have fun untangling that. Why not stop this bullshit and just transition to IPv6??
- jandrese 2y ago
  I'm not too worried about the Internet at large actually using Class E addresses. IPv4 is considered obsolete so there shouldn't be any "future use" coming out of the standards committee.
- unilynx 2y ago
  class E can't ever be used on the public internet anyway. too many existing firewalls are already configured to drop anything with a source address in range 224.0.0.0-255.255.255.255 (class D and E)
- Hikikomori 2y ago
  There's an RFC to use it in the Internet, unlikely to be ratified though. Usually because your provider does not fully support IPv6 yet, which is the case at least with Azure and AWS, likely Google as well.
sulandor 2y ago
amazon (among others) is doing this for some time rfc's have been written and forgotten, please calm down
ay 2y ago
“ As mentioned in Google VPC network valid IPv4 ranges, Class E addresses (240.0.0.0/4) are reserved for future use, as noted in RFC 5735 and RFC 1112 — however, that doesn’t mean you can’t use them today in certain circumstances.” Wow. reserved means “kept aside”. Once someone starts using them, they stop being kept aside. This means that de-facto they are private addresses now. I suppose it’s a pragmatic choice. But, wow, so much for having interactions in relevant standards bodies, multistakeholder engagements, etc. Why bother. Classy. /s
ffhcx 2y ago
y

28 points by l1n 2y ago

deleted 2y ago

[deleted]

> However not all enterprises or applications are ready for IPv6 yet. I don't think this statement will ever be false in my lifetime.

ddtaylor 2y ago

Anything but IPv6 it would seem.

deleted 2y ago
[deleted]
ianburrell 2y ago
I wish that Kubernetes had started as IPv6-only or IPv6-first. IPv6 is perfect for giving each pod its own address. There is more space for giving each host a subnet, or giving each pod its own address in the subnet. That would have gotten rid of the CNI overlay networks. It would also made barrier between the internal and external networks. It would require running NAT64 and DNS64 in most clusters.
- magicalhippo 2y ago
  > I wish that Kubernetes had started as IPv6-only or IPv6-first Seems a bit strange to me that they didn't, given how you'll want to use some sort of ingress anyway, so IPv4 could be delegated to those edge points.
tialaramex 2y ago
> "It is remarkable, love," he said, looking at Nell for the first time, "how much money you can make shovelling back the tide." [ "Diamond Age" by Neal Stephenson ]

jandrese 2y ago

> Snap initially considered migrating to IPv6, but concerns about application readiness and interoperability led them to adopt dual-stack GKE nodes and GKE pods (IPv6 + Class E IPv4). This solution mitigated IP exhaustion and provided Snap with multiple years of IP address scale needed to support future growth and reduce operational overhead. In addition, this approach also aligned with Snap’s long-term strategy for migrating to IPv6. Yet another horrible hack to avoid having to actually learn IPv6.

dmitrygr 2y ago
The first time someone calls you a horse, you punch him on the nose; the second time someone calls you a horse, you call him a jerk; but the third time someone calls you a horse, well then, perhaps it's time to go shopping for a saddle.
yjftsjthsd-h 2y ago
How does using both avoid learning v6?
- jandrese 2y ago
  Years of experience with "dual stack" organizations that completely neglect the IPv6 side or do stuff like install MITM firewalls that can't handle IPv6 so the addresses are only useful internally.
Hikikomori 2y ago
Who says they don't want to learn? When your provider doesn't support IPv6-only for all your purposes (I don't know about Google but Azure and AWS doesn't do that) then you'll just exhaust RFC1918 and not work on a solution for it?

hypeatei 2y ago

Interesting read, but it seems like a waste of time to keep finding little tricks and band aids with IPv4 when v6 exists. > However not all enterprises or applications are ready for IPv6 yet Ah yeah, there it is. Please, just fucking prioritize upgrading to IPv6 and be done with it. Frustrating. Cloud providers need to hurry up as well, Azure still doesn't fully support v6 on their app services (web servers) either. It's in public preview but has been a roadmap item for longer. It also comes with certain caveats like what tiers can use it.

dangerlibrary 2y ago

It never made sense to me why the authors of the IPv6 standard chose to use an entirely different address format, instead of extending address space in a backwards-compatible way. Why not just append/prepend the 12 additional bytes onto existing IPv4 addresses, and write the standard such that all valid IPv4 addresses are also valid IPv6 addresses? I'm not an RFC author, but something like "All existing IPv4 addresses will be reachable under the 0.0.0.0.0.0.0.0.0.0.0.0 prefix in the IPv6 Standard" seems like it would've made migration relatively trivial. The draft standard is 26 years old. The official standard is 7 years old, and we are still reading articles about how "not all enterprises or applications are ready for IPv6 yet." This question is coming from a place of genuine confusion and curiosity - I really don't get it. Did the authors of the standard just assume that migration and adoption would be easier than they've turned out to be? Was it a fairness issue where somehow this would have granted dominion over huge swaths of the new address space to existing players?

ianburrell 2y ago
Because writing the addresses is not the important part. You can write IPv6 addresses as dotted decimal. The length is the problem, IPv6 format is more compact. Addresses are not the hard part in upgrading. The embedding doesn't help with compatibility. IPv4 still can't access IPv6. IPv6 can't access IPv4. It actually breaks NAT64, which depends on special prefix. Also, extending IPv4 address space has the problem that bake the misallocation into IPv6. IPv4 is broken in the small chunks that makes the routing table large. It also means that new organizations will have a hard time getting address space cause they need to get IPv4 addresses.
- dangerlibrary 2y ago
  "The length is the problem, IPv6 format is more compact." "IPv4 still can't access IPv6. IPv6 can't access IPv4." I guess my point is that these statements represent choices made by humans. Leaving the decimal representation aside, I don't get why they made these choices. If every IPv4 address were a valid IPv6 address, then these statements wouldn't be true.

tgma 2y ago

djb on IPv6 incompatibility being a bad design choice: https://cr.yp.to/djbdns/ipv6mess.html https://cr.yp.to/djbdns/ipv6mess.html

kstrauser 2y ago
His points were wrong 22 years ago, and they continue to be wrong today. It's a poor document to cite as a rationale for avoiding IPv6.
- tgma 2y ago
  Huh? It is not a rationale for avoiding IPv6. Quite the contrary. It is a critique on how IPv6 was designed without being interoperable with IPv4 address space. It is very clear that he was right given there are 22 more years of evidence of IPv4 still being very much with us, and foreseeable future.
  - ay 2y ago
    Quote from the doc: “ Answer: We go through every place that 4-byte IPv4 addresses appear, and allow 16-byte IPv6 addresses in the same place.” - this text is pretty much the definition of “IPv6 transition”.
    - throw0101d 2y ago
      Also from the document: > Unfortunately, instead of simply allowing 16-byte A records, people introduced new ``AAAA records'' into the DNS protocol, creating several unnecessary complications in DNS software. "Simply". Yeah. Until some non-upgraded client gets an 'extended-A' record response that's the "wrong" size (according itself) and drops the packet (which is the sane thing to do security-wise) and then you've just broken your DNS completely: IPv4 no longer works for some people, and neither does IPv4+. So you have to wait until all client software understands extended-A records—but what's the motivation for enabling new client software if no one is enabling server-side records because there's no client software. Chicken-egg. Which is one argument against IPv6: why enable client-side when there's no content, and why bother server-side if there are no client requests? Chicken-egg.

solatic 2y ago

Incredibly unsafe. If Class E is reserved "for future use", maybe that could include public use. The first rule of hardening production reliability is, make sure it doesn't just work, but that it will continue to work. Tomorrow, some customer of yours gets assigned addresses from Class E. Have fun untangling that. Why not stop this bullshit and just transition to IPv6??

jandrese 2y ago
I'm not too worried about the Internet at large actually using Class E addresses. IPv4 is considered obsolete so there shouldn't be any "future use" coming out of the standards committee.
unilynx 2y ago
class E can't ever be used on the public internet anyway. too many existing firewalls are already configured to drop anything with a source address in range 224.0.0.0-255.255.255.255 (class D and E)
Hikikomori 2y ago
There's an RFC to use it in the Internet, unlikely to be ratified though. Usually because your provider does not fully support IPv6 yet, which is the case at least with Azure and AWS, likely Google as well.

sulandor 2y ago

amazon (among others) is doing this for some time rfc's have been written and forgotten, please calm down

ay 2y ago

“ As mentioned in Google VPC network valid IPv4 ranges, Class E addresses (240.0.0.0/4) are reserved for future use, as noted in RFC 5735 and RFC 1112 — however, that doesn’t mean you can’t use them today in certain circumstances.” Wow. reserved means “kept aside”. Once someone starts using them, they stop being kept aside. This means that de-facto they are private addresses now. I suppose it’s a pragmatic choice. But, wow, so much for having interactions in relevant standards bodies, multistakeholder engagements, etc. Why bother. Classy. /s

ffhcx 2y ago