12 ms·

AWS and their billions in IPv4 addresses

192 points by bgpdude 6y ago

dogma1138 6y ago
It will be quite interesting to see how these assets will be devaluated in the future when we finally roll off IPv4 (if we ever do). Besides the direct technical benefits of having so many addresses Amazon also potentially holds billions worth of assets that will eventually devalue and devalue quite extremely that they could write off for tax purposes. IPv4 addresses will continue to appreciate in value until their collapse which means that Amazon might be able to write off tenfold than what they paid to acquire them in the first place.
- sushicalculus 6y ago
  EDIT: Looks like I was wrong when I thought IPv6 was backwards compatible IPv6 is backwards compatible so why would we "finally roll off IPv4"? IPv6 has been "here" in production for over a decade, and has seen slow adoption. I suspect it may be another 8-10 years until IPv6 gets close to a majority, but wouldn't IPv4 still be in-use? What would force a roll off of IPv4? Either way, I too would like to know more about how IPv4 space has appreciated/depreciated in the past decade
  - lucb1e 6y ago
    In what way is it backwards compatible? It's not as if all allocations are kept and people just add 96 bits to their existing addresses. They are completely disjoint address spaces and protocols, as compatible as Tor is with the general internet: to communicate between the two, you need a proxy.
    - mulmen 6y ago
      Tor operates over the Internet. Did you mean to compare Tor to the World Wide Web?
      - eznzt 6y ago
        Tor is sort of a virtual network. You need a gateway between the virtual network and the Internet to access the Internet.
        mulmen 6y ago
        Tor overlays the Internet. It can’t operate without that underlying network. That is not how IPv6 works relative to IPv4.
        lucb1e 6y ago
        Yeah, that is indeed a flaw in the comparison.
    - sushicalculus 6y ago
      I was wrong. I must have rememberd an early draft or the translation layer
    - Dylan16807 6y ago
      So I'm not saying it's backwards compatible, exactly, but what would you call 64:ff9b::?
      - mulmen 6y ago
        A reserved gateway address space? https://en.m.wikipedia.org/wiki/NAT64 https://en.m.wikipedia.org/wiki/NAT64
        Dylan16807 6y ago
        By embedding the IPv4 address space inside the IPv6 address space. They are different protocols but they are not disjoint address spaces. You add 96 bits to the address and boom it's converted.
  - otterley 6y ago
    IPv6 is not backwards compatible - it is a completely separate networking stack for all devices. And routers that speak only IPv4 have no idea what to do with IPv6 traffic. Similarly, IPv6-only endpoints cannot communicate with IPv4 endpoints without some sort of 6-to-4 gateway between them.
    - bluGill 6y ago
      There shouldn't be any IPv4 only routers in the wild anymore. That isn't to say they aren't out there, only that anyone who has one is negligent in not updated/replacing it within the last 15 years or so. IPv6 is not a new thing, it was already live in 1996 (my memory might be a little off, but not much). Maybe you have turned off IPv6, but it should be supported and any competent IT should have a plan to turn it on (it might be a 5 year plan without any budget, but the plan should exist). Likewise you should have a plan to turn off IPv4 (if this plan has a budget it should be paid for entirely by the sale of your existing IPv4 block - this is mostly the business costs of IPv4 only customers being unable to reach you NOT the technical cost of updating all your servers)
  - linuxftw 6y ago
    The problem isn't supporting IPv6, the problem is often having to support both 4 and 6 simultaneously. There's just a ton of work to get a single IP stack working on infrastructure, and it's more than twice the work to get 2 working in tandem.
    - CountSessine 6y ago
      Is it really? In 2020? What examples are there? I worked on network switches at a major switch manufacturer and our IPv6 setup mostly just came along for the ride (or at least I thought it did - maybe our customers thought differently!) (Genuinely interested in hearing war stories from the front on this)
      - linuxftw 6y ago
        Just look at the amount of engineering effort and time that has gone into projects like kubernetes: https://github.com/kubernetes/enhancements/issues/508 https://github.com/kubernetes/enhancements/issues/508 That's just for IPv6 support, which is beta, and took years to get there. Dual Stack is still in alpha, and that didn't appear until a really recent release (1.16). I think routers and switches are probably the easier pieces of the puzzle, at least as far as managing them goes. The vendors have worked out all the kinks (hopefully?) before the equipment gets to you. That's just to get the stuff to work. Then you have corporate/government policies and validation. Then you have to solve problems like "My VM resolves things to IPv6 by default, but I have no IPv6 gateway so everything times out". And then make sure that logic makes it up through your entire stack. Multicast? Not allowed on many networks. Is it the end of the world? No. It's just a lot of extra work for everyone.
        CountSessine 6y ago
        Yeah - that’s been my intuition, too. First we got software support in switches. Then OS network stacks started to support it. Then we got hardware switching chips that supported it. But the application and deployment layers just seem to be super hard. Just look at how hard it’s been to get IPv6 on AWS. Someday, hopefully!
  - xnyan 6y ago
    >IPv6 is backwards compatible It would be really really great if true, unfortunately this is not correct. IPv6 SHOULD have been this way, but The Powers That Be took it as an opportunity to "correct" other IPv4 issues and now we have what we have.
    - TheDong 6y ago
      > IPv6 SHOULD have been this way How would it have possibly been backwards compatible? Plenty of routers and ip-aware switches have, in hardware, specified that ips are 32-bits, so anything that added more bits would necessarily break existing hardware, and thus not be backwards compatible. That's not to mention all the software that has similarly hardcoded the number of bytes in an ip. How could we possibly have made ipv6 backwards compatible?
      - bluGill 6y ago
        I worked on the first switch ever sold in 1995 - it didn't support IPv6, but I assure you all the engineers working on it were aware of IPv6 and assumed IPv4 would be long gone by today. For reference back then we were more concerned about how the switch handled IPX (Novell Netware) than IP. Everything else since then was designed in the era where IPv6 is coming soon enough that you ensure you can support it with a software update if you don't support it. That isn't to say IPv6 couldn't have been done in a backward compatible way. I can think of ways to do that, and dozens of pros and cons - even though I haven't been in networking for 20 years and so I've forgotten a lot.
        TheDong 6y ago
        > ensure you can support it with a software update if you don't support it For hardware, that doesn't sound that different from not supporting it. Convincing end-users of internet hardware to update their switch's firmware is hard. Ubiquiti has done a pretty good job of making updates actually doable, but for most hardware I doubt it'd happen more quickly than the hardware itself would fail. > That isn't to say IPv6 couldn't have been done in a backward compatible way. I can think of ways to do that Can you explain any of those ways?
  - Arnt 6y ago
    The only realistic reason to get rid of IPv4 is its unreliability, if significant. 15-20 years ago IPv6 routing was often tunneled over v4, and it was noticeably less reliable than v4. Nowadays v6 is a touch faster/more reliable than v4, not very much. If that difference widens much and most users have v6 anyway, then v4 might get the axe. Otherwise... I don't think so. FWIW I already operate some v6-only services. Not for the general audience. An SSH bastion host, a backup service, that kind of thing, not web sites for the general audience. I'm not in the least surprised if some warez topsite is v6-only already.
- throwawaytemp27 6y ago
  That is not how accounting works. You can only write off the book value (ie what you paid).
  - formercoder 6y ago
    And GAAP would have to let you capitalize them. Not sure if they do.
    - unilynx 6y ago
      But if they weren't allowed to capitalize them, the addresses would have already been written off at time of purchase, so the ipv6 switch wouldn't hurt them anyway
- OJFord 6y ago
  For that tax 'advantage' you'd have to buy them when they were supremely expensive before collapse, so that you had the high acquisition cost of a now-worthless asset to write off. But you'd have paid X for a reduction of X in tax liability, which, assuming you pay a sub-100% marginal tax rate, is a strictly dumb idea.
  - deleted 6y ago
    [deleted]
- jacobsenscott 6y ago
  I would bet NAT and reclaiming of unused address space will keep the status quo going for another 20 years at least. Especially since computing is migrating to a few huge data centers where almost all the communication is over their internal private network.
  - CountSessine 6y ago
    That’s sort of what I’ve come to believe. Imagine an internet where there are two types of IPv4 addresses assigned. The server addresses belong to cloud data providers and are provided as elastic IPs and are used to host public websites - any kind of website that you’re likely to type into an URL bar. Client addresses are owned by ISPs and are the routable NAT addresses with thousands of customers sharing them. A few client addresses are provided to ISP customers that are willing to pay a monthly fee. It’s ugly but it’s kind of the sort of system that everyone is incentivized to move to. Which kind of makes me sad.
sp332 6y ago
I bet that /8 went for more than the "average" value of an IP address. Larger contiguous blocks are more useful, and having a /8 has got to have some prestige value as well.
- lucb1e 6y ago
  But on the other hand, wholesale items are rarely sold at piece prices. I'm not sure the price increased that much just by having a large block. Are there other big blocks sold for known prices?
  - bgpdude 6y ago
    the only recent one i'm aware of is this one: http://www.southgatearc.org/news/2020/october/sale-of-amateur-radio-amprnet-tcp-ip-addresses.htm http://www.southgatearc.org/news/2020/october/sale-of-amateu... AWS paid $108 million for this /10. That’s $25.74 per IP address.
- perlgeek 6y ago
  I've talked to our resident backbone engineer about that about a year ago, and he said that larger blocks transfer for a lower value per IP, on average. I was surprised by that, but he showed me data that supported that. Update: I just found http://www.circleid.com/posts/20180731_the_ipv4_market_2018_mid_year_report/ http://www.circleid.com/posts/20180731_the_ipv4_market_2018_... which says > As we head into the last half of 2018, small blocks are selling for just over $18/number, mid-blocks are in the $15-$18/number range, and large blocks have surpassed the $20/number threshold so it seems there's a bit of an U shape, where mid-sized blocks have the lowest price per address.
  - toast0 6y ago
    I think this makes sense. A lot of networks need a few addresses, so /24 - /22 is probably pretty active, and you have to justify the addresses, so someone who really only needs a /24 isn't able to buy a /21 and make it work; very few people need huge swaths, but those will be able to really use them and may value having them contiguous. In the middle, it doesn't make much difference having a /16 or two /17s or four /18s, so there's a lot more flexibility, and you take the lowest price per IP if there's a block you can use.
beckler 6y ago
I was working at GE when AWS bought GE's 3.0.0.0/8 block, and it caused a massive headache. GE's network is gigantic, and there are a ridiculous number of assets. They didn't give much notice about the sale, so they shimmed all of the routing internally, but lots of services still pointed to the block. There supposedly was an agreement that Amazon wouldn't start provisioning them for a certain amount of time, but whatever amount of time was specified, they either didn't honor it, or it wasn't enough time. We were also moving assets over to AWS, and all of these things going on simultaneously caused what we called the three-pocalypse. We would occasionally run across issues with external sites or newly provisioned lambdas who were on Amazon's new 3.0.0.0/8 block, but we couldn't reach them because internally that IP address didn't exist. At the same time, they would open up a small block to allow access to those external sites, and then some internal service would no longer respond. Repeat ad nauseam. It was also compounded by the fact that there are countless teams in GE and not everyone would connect with who made what changes.
- RKearney 6y ago
  Sounds like the perfect time to migrate to IPv6!
  - sroussey 6y ago
    IP6 on AWS ECS is a pain, and impossible if you use their recommended and dedicated command line tool.
    - Ericson2314 6y ago
      This has always seemed deeply suspicious to me. It's like AWS is trying to prop up the value of its speculation on IPv4 addresses.
      - leetcrew 6y ago
        or merely lazy. why spend time/$$$ supporting IPv6 when you have plenty of IPv4 addresses?
ryan_j_naughton 6y ago
Can someone explain why these have such value? What systems cannot handle IPV6 yet?
- casperb 6y ago
  Like 60-70% of all internet users: https://www.google.com/intl/en/ipv6/statistics.html#tab=ipv6-adoption&tab=ipv6-adoption https://www.google.com/intl/en/ipv6/statistics.html#tab=ipv6...
  - Spivak 6y ago
    This is a little misleading. This just measures users that are using IPv6 not users that would be able to use IPv6 if IPv4 stopped working. My local network and ISP are perfectly capable of using IPv6 but you have to call them to switch.
  - jcranmer 6y ago
    Zoom into 2020, and you can clearly see when the lockdowns due to the pandemic start on that chart. The other interesting artifact is that it looks like IPv6 network rollout tends to almost exclusively happen in the April-July timeframe.
- gertrunde 6y ago
  Imagine that you have two ways of addressing your system. 1) Allows you to access almost 100% of your potential customers. 2) Allows you to access around 35% of your potential customers. Which one would you choose any why?
  - ArchOversight 6y ago
    Both. Especially if that one with 35% of my customers provides me with lower latency, higher throughput and costs me less in CPU time/power to run my traffic across. The rest of the people I need to eat the cost for...
    - 6y ago
vmception 6y ago
The addressing apocalypse that never comes
- remus 6y ago
  Presumably there will come a point where the cost of purchasing enough ipv4 addresses outweighs the cost of making a project work with ipv6. It'll be an interesting piece of economics to see where that boundary is.
  - wmf 6y ago
    The economics (for consumer ISPs) are explored at http://www.asgard.org/documents.html http://www.asgard.org/documents.html
- jedberg 6y ago
  Carrier grade NAT and cell phones being IPv6 only helped stem the tide a lot.
  - skipnup 6y ago
    And my cell phone connection in Germany is IPv4 only :-(
    - usr1106 6y ago
      Well, Germany, they don't even have cell phone coverage in every village. And where they have it data rates can be abysmal. Yesterday the news hit the mainstream media that Nokia got a NASA deal to build 4G on the moon. Couldn't resist thinking building it in Germany would be more useful. Disclaimer: I lived in the county over 30 years and still have strong links.
  - eznzt 6y ago
    I don't think "cell phones being IPv6 only" is a widespread phenomenon. If anything, "cell phones being CGNAT only" is the most popular case.
- 6y ago
mmm_grayons 6y ago
It's interesting when you look at it from this perspective. AWS and other big IPv4 owners have a financial incentive to hold back IPv6 deployment in a convenient and scalable fashion, because that would quickly wipe out billions in value of a scarce resource. It's like some people who own NFA firearms: they support keeping the regulations because a lightning link isn't worth $15k without it, it's just a cheap piece of stamped sheet metal. Edit: it's also an easy way to outcompete others; IPv4 space has become a huge capex for starting a cloud provider.
- bluGill 6y ago
  Until one runs out of IPv4 addresses. I think it is safe to say that Amazon, Facebook, and google each have data centers big enough that IPv4 address management is a headache. I believe Facebook is internally IPv6 (that is server to server communications - their public space obviously has IPv4, and their desktop space can get by with NAT and IPv4 if they want to). Google has done enough promoting of IPv6 that I think they are the same. I'm not sure what AWS is - they sell server time so they have more need for IPv4 to the servers (google cloud has the same concerns)
  - ArchOversight 6y ago
    Facebook only has IPv4 at the edge. Everything internal is IPv6.
    - samoa42 6y ago
      suggests that ipv6 is here to replace the rfc1918 space but "the internet" largely remains ipv4. the high v6 penetration in mobile networks likewise suggests that smartphones are generally not a part of the network as much as dedicated leaf nodes that require no inbound connections.
      - p1mrx 6y ago
        There's a difference between "IPv4 is only present at the edge" and "only IPv4 is present at the edge". I think Facebook is doing IPv4+IPv6 at the edge, and IPv6-only internally.
iJohnDoe 6y ago
It’s a good future-proofing investment by AWS. IPv4 is not going away anytime soon. We’ll be dependent on IPv4 for a long time. IPv4 scarcity only hits in certain segments. It doesn’t hit the major players. The real problem is the hoarding by the major players. AWS will most likely never use that many IP addresses. I know that sounds like a stretch based on what we’re led to believe about running out of IPv4 addresses. I don’t know their IPv4 utilization, so it’s a guess. My guess is based on how we deploy internet facing services these days. It’s less dependent on needing lots of public IPs.
- labawi 6y ago
  IMO it's good for the internet if few entities were to hoard most IPv4 addresses. Only while IPv4's are sufficiently available will we keep using IPv4-only networks and keep prolonging our miseries. My ISP only gives CGNAT or is happy to rent IPv4 for a premium, one address at a time. Keeping IPv6 out helps the IPv4 margins and staves off competition.
- shiftpgdn 6y ago
  According to the article AWS usage is at 47% of their IP pool.
- kaliszad 6y ago
  You cannot use IPv4 100% efficiently. Even if you approach 90% usage, that would probably be very optimistic. AWS has probably millions of physical servers by now (they had ca. 1,4 million servers by December 2014 it seems) each with probably hundreds of services each needing an IP. Yes, you can stretch 10.0.0.0/8 with NATs all over the place but at some point, you need the public IP. With that kind of appetite you need multiple /8s of IPv4 space just to keep things running. I don't think they are crazy buying that much address space and concurrently improving IPv6 support - they most likely really need it.
  - tudelo 6y ago
    > You cannot use IPv4 100% efficiently. Can you expand on this a bit? I'm not a networking guru but it's my understanding that an IP address is an IP address, how could it be used more or less efficient?
Ecco 6y ago
The sad thing is that's probably why IPv6 is never going to be a thing. Why would all major tech companies be willing to write off such expensive assets?
- lolc 6y ago
  They can try and resist. But they will lose. For example: I've set up a IP6-only backend because it was easier than getting IP4-connectivity. And when I tried Travis and they couldn't connect to that backend, I dropped Travis immediately. That is still uncommon, but I'm sure I'm not the only one who's made a decision in that direction.
- tptacek 6y ago
  Because they routinely do? They're tech companies. Everything they own depreciates.
- deleted 6y ago
  [deleted]
valenciarose 6y ago
The artificial resource constraints created by IPv4 are a substantial barrier to entry. Prediction: we’re going to see a court case that forces a mandatory retirement of IPv4 sometime in the next 15 years.
- lolc 6y ago
  I think that IP4 will see itself out when service degradation becomes generally noticeable. CGNAT is really painful to operate. Remember the IE6 banners? There will be a time in ten years (haha it's always ten years!) when you'll see "Our site runs with degraded performance over IPv4. Please contact your administrator."
  - anderspitman 6y ago
    > CGNAT is really painful to operate It also makes it much more difficult for customers to stress your upload bandwidth though.
    - lolc 6y ago
      You mean to say customers get a degraded experience on CGNAT.
      - anderspitman 6y ago
        Yup. If you can't host a webserver from your home IP, that's one less thing your ISP needs to worry about taxing their network. It's sad the incentives just aren't aligned for a decentralized internet.
  - eznzt 6y ago
    >CGNAT is really painful to operate. For whom? There are lots of providers (for example Huawei) that offer turnkey solutions for ISPs to painlessly roll out CGNAT.
    - lolc 6y ago
varenc 6y ago
MIT used to have all of 18.0.0.0/8 but they sold half of it to AWS a few years ago. Because MIT had so many IP addresses they didn't need to use a NAT. Everyone connecting over wifi got a real world 18.* IP address. It also made network architecture easy. Every MIT building got its own B class subnet. Hilariously, even the MIT boat house which couldn't have had more than a dozen computers online at once had 65,534 addresses. It made it very easy to find out which building someone was in based on their IP.
- deleted 6y ago
  [deleted]
- angulardragon03 6y ago
  My old uni still has two /16s. Everyone on campus gets their own “public” IP (behind the firewall, of course).
- icedchai 6y ago
  IP blocks were handed out like candy in the mid-90's and earlier. I have a /24 ("class C") personally, and know of several others who do also.
- stordoff 6y ago
  My old university has three /16s, and a bunch of other IP blocks, so gave public IPs to everyone (though they now seem to be using private IPs for some purposes). I never looked at how the public machines were allocated, but it did make picking out traffic from the residences trivial - they reverse DNS resolved to <userid>.<college.>cam.ac.uk
unklefolk 6y ago
What does the slash mean in the article? Where it says “... in 2018 GE sells 3.0.0.0/8” what does the “/8” mean? I guess it indicates the range of addresses?
- wging 6y ago
  Yep. https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing#CIDR_notation https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing... for more info.
- codegladiator 6y ago
  https://www.calculator.net/ip-subnet-calculator.html https://www.calculator.net/ip-subnet-calculator.html Checkout the table on the page (scroll down)
- deleted 6y ago
  [deleted]
- SheepSlapper 6y ago
  You're right that it indicates a range of addresses, the number after the slash is the number of starting bits that the addresses all share. Check out CIDR Blocks for a more complete answer: https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing#CIDR_blocks https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing...
- justusthane 6y ago
  /8 is the same as a 255.0.0.0 subnet mask. Also known as a class A subnet. In other words, literally any IP address that starts with “3.” In other other words, 16,777,216 IP addresses. In other other other words, about half a percent of the total number of public IPv4 addresses. Crazy.
mbreese 6y ago
Id love to know similar stats on IPv4 space for Google Cloud and Azure.
neiman 6y ago
If we insist on IPv4, at least treat those IPs as a public resource of humanity, and don't just let anyone horde them in all kinds of strange deals.
- wmf 6y ago
  Officially, hoarding has never been allowed. You can't buy IPs unless you have a plan to use them. And there's nothing strange about a company with tens of millions of VMs buying tens of millions of addresses.
  - neiman 6y ago
    People hoarded toilet paper when Corona began with full intention to use them. People hoard food in wartime with intention to eat it. So yes, Amazon hoard IPv4 with intention to maybe use them in the future. But hording it is, and in a non-hoard system they'd rent - not buy - IPv4 with a month notice or so.
anderspitman 6y ago
I've toyed with the idea of buying a /24 as a (probably terrible) investment that I would actually get at least some use out of in the short term (some VPSes like vultr let you bring your own IPs). Anyone have enough experience with this to talk me out of it?
- Thorentis 6y ago
  With my investing luck, the world would suddenly switch to 100% IPv6 the day after the sale went through.
  - mikestew 6y ago
    If that's all it takes, I'm willing to start the co-op that fronts you the money.
- bibabaloo 6y ago
  In a similar vein, is there a cost effective way to purchase a single ipv4 address? Just for the novelty I think it'd be cool if I could spend ~$100 to have an ipv4 address that's permanently mine.
  - jlgaddis 6y ago
    I've got plenty I'm not using, I'll give you one. Here, this one's an easy one to remember, you can have it: 127.1. Enjoy!
  - fanf2 6y ago
    The minimum IPv4 allocation is a /24 because that is the smallest network that can be globally routed. Smaller networks are filtered out of BGP to stop routing tables getting too big too fast. Anything smaller than /24 has to be rented from a network provider who will advertise an aggregate route covering many customers.
- pera 6y ago
  AFAIK you need to demonstrate a real need to actually acquire a block, this is not like buying a domain name.

192 points by bgpdude 6y ago

It will be quite interesting to see how these assets will be devaluated in the future when we finally roll off IPv4 (if we ever do). Besides the direct technical benefits of having so many addresses Amazon also potentially holds billions worth of assets that will eventually devalue and devalue quite extremely that they could write off for tax purposes. IPv4 addresses will continue to appreciate in value until their collapse which means that Amazon might be able to write off tenfold than what they paid to acquire them in the first place.

sushicalculus 6y ago
EDIT: Looks like I was wrong when I thought IPv6 was backwards compatible IPv6 is backwards compatible so why would we "finally roll off IPv4"? IPv6 has been "here" in production for over a decade, and has seen slow adoption. I suspect it may be another 8-10 years until IPv6 gets close to a majority, but wouldn't IPv4 still be in-use? What would force a roll off of IPv4? Either way, I too would like to know more about how IPv4 space has appreciated/depreciated in the past decade
- lucb1e 6y ago
  In what way is it backwards compatible? It's not as if all allocations are kept and people just add 96 bits to their existing addresses. They are completely disjoint address spaces and protocols, as compatible as Tor is with the general internet: to communicate between the two, you need a proxy.
  - mulmen 6y ago
    Tor operates over the Internet. Did you mean to compare Tor to the World Wide Web?
    - eznzt 6y ago
      Tor is sort of a virtual network. You need a gateway between the virtual network and the Internet to access the Internet.
      - mulmen 6y ago
        Tor overlays the Internet. It can’t operate without that underlying network. That is not how IPv6 works relative to IPv4.
        lucb1e 6y ago
        Yeah, that is indeed a flaw in the comparison.
  - sushicalculus 6y ago
    I was wrong. I must have rememberd an early draft or the translation layer
  - Dylan16807 6y ago
    So I'm not saying it's backwards compatible, exactly, but what would you call 64:ff9b::?
    - mulmen 6y ago
      A reserved gateway address space? https://en.m.wikipedia.org/wiki/NAT64 https://en.m.wikipedia.org/wiki/NAT64
      - Dylan16807 6y ago
        By embedding the IPv4 address space inside the IPv6 address space. They are different protocols but they are not disjoint address spaces. You add 96 bits to the address and boom it's converted.
- otterley 6y ago
  IPv6 is not backwards compatible - it is a completely separate networking stack for all devices. And routers that speak only IPv4 have no idea what to do with IPv6 traffic. Similarly, IPv6-only endpoints cannot communicate with IPv4 endpoints without some sort of 6-to-4 gateway between them.
  - bluGill 6y ago
    There shouldn't be any IPv4 only routers in the wild anymore. That isn't to say they aren't out there, only that anyone who has one is negligent in not updated/replacing it within the last 15 years or so. IPv6 is not a new thing, it was already live in 1996 (my memory might be a little off, but not much). Maybe you have turned off IPv6, but it should be supported and any competent IT should have a plan to turn it on (it might be a 5 year plan without any budget, but the plan should exist). Likewise you should have a plan to turn off IPv4 (if this plan has a budget it should be paid for entirely by the sale of your existing IPv4 block - this is mostly the business costs of IPv4 only customers being unable to reach you NOT the technical cost of updating all your servers)
- linuxftw 6y ago
  The problem isn't supporting IPv6, the problem is often having to support both 4 and 6 simultaneously. There's just a ton of work to get a single IP stack working on infrastructure, and it's more than twice the work to get 2 working in tandem.
  - CountSessine 6y ago
    Is it really? In 2020? What examples are there? I worked on network switches at a major switch manufacturer and our IPv6 setup mostly just came along for the ride (or at least I thought it did - maybe our customers thought differently!) (Genuinely interested in hearing war stories from the front on this)
    - linuxftw 6y ago
      Just look at the amount of engineering effort and time that has gone into projects like kubernetes: https://github.com/kubernetes/enhancements/issues/508 https://github.com/kubernetes/enhancements/issues/508 That's just for IPv6 support, which is beta, and took years to get there. Dual Stack is still in alpha, and that didn't appear until a really recent release (1.16). I think routers and switches are probably the easier pieces of the puzzle, at least as far as managing them goes. The vendors have worked out all the kinks (hopefully?) before the equipment gets to you. That's just to get the stuff to work. Then you have corporate/government policies and validation. Then you have to solve problems like "My VM resolves things to IPv6 by default, but I have no IPv6 gateway so everything times out". And then make sure that logic makes it up through your entire stack. Multicast? Not allowed on many networks. Is it the end of the world? No. It's just a lot of extra work for everyone.
      - CountSessine 6y ago
        Yeah - that’s been my intuition, too. First we got software support in switches. Then OS network stacks started to support it. Then we got hardware switching chips that supported it. But the application and deployment layers just seem to be super hard. Just look at how hard it’s been to get IPv6 on AWS. Someday, hopefully!
- xnyan 6y ago
  >IPv6 is backwards compatible It would be really really great if true, unfortunately this is not correct. IPv6 SHOULD have been this way, but The Powers That Be took it as an opportunity to "correct" other IPv4 issues and now we have what we have.
  - TheDong 6y ago
    > IPv6 SHOULD have been this way How would it have possibly been backwards compatible? Plenty of routers and ip-aware switches have, in hardware, specified that ips are 32-bits, so anything that added more bits would necessarily break existing hardware, and thus not be backwards compatible. That's not to mention all the software that has similarly hardcoded the number of bytes in an ip. How could we possibly have made ipv6 backwards compatible?
    - bluGill 6y ago
      I worked on the first switch ever sold in 1995 - it didn't support IPv6, but I assure you all the engineers working on it were aware of IPv6 and assumed IPv4 would be long gone by today. For reference back then we were more concerned about how the switch handled IPX (Novell Netware) than IP. Everything else since then was designed in the era where IPv6 is coming soon enough that you ensure you can support it with a software update if you don't support it. That isn't to say IPv6 couldn't have been done in a backward compatible way. I can think of ways to do that, and dozens of pros and cons - even though I haven't been in networking for 20 years and so I've forgotten a lot.
      - TheDong 6y ago
        > ensure you can support it with a software update if you don't support it For hardware, that doesn't sound that different from not supporting it. Convincing end-users of internet hardware to update their switch's firmware is hard. Ubiquiti has done a pretty good job of making updates actually doable, but for most hardware I doubt it'd happen more quickly than the hardware itself would fail. > That isn't to say IPv6 couldn't have been done in a backward compatible way. I can think of ways to do that Can you explain any of those ways?
- Arnt 6y ago
  The only realistic reason to get rid of IPv4 is its unreliability, if significant. 15-20 years ago IPv6 routing was often tunneled over v4, and it was noticeably less reliable than v4. Nowadays v6 is a touch faster/more reliable than v4, not very much. If that difference widens much and most users have v6 anyway, then v4 might get the axe. Otherwise... I don't think so. FWIW I already operate some v6-only services. Not for the general audience. An SSH bastion host, a backup service, that kind of thing, not web sites for the general audience. I'm not in the least surprised if some warez topsite is v6-only already.
throwawaytemp27 6y ago
That is not how accounting works. You can only write off the book value (ie what you paid).
- formercoder 6y ago
  And GAAP would have to let you capitalize them. Not sure if they do.
  - unilynx 6y ago
    But if they weren't allowed to capitalize them, the addresses would have already been written off at time of purchase, so the ipv6 switch wouldn't hurt them anyway
OJFord 6y ago
For that tax 'advantage' you'd have to buy them when they were supremely expensive before collapse, so that you had the high acquisition cost of a now-worthless asset to write off. But you'd have paid X for a reduction of X in tax liability, which, assuming you pay a sub-100% marginal tax rate, is a strictly dumb idea.
- deleted 6y ago
  [deleted]
jacobsenscott 6y ago
I would bet NAT and reclaiming of unused address space will keep the status quo going for another 20 years at least. Especially since computing is migrating to a few huge data centers where almost all the communication is over their internal private network.
- CountSessine 6y ago
  That’s sort of what I’ve come to believe. Imagine an internet where there are two types of IPv4 addresses assigned. The server addresses belong to cloud data providers and are provided as elastic IPs and are used to host public websites - any kind of website that you’re likely to type into an URL bar. Client addresses are owned by ISPs and are the routable NAT addresses with thousands of customers sharing them. A few client addresses are provided to ISP customers that are willing to pay a monthly fee. It’s ugly but it’s kind of the sort of system that everyone is incentivized to move to. Which kind of makes me sad.

sp332 6y ago

I bet that /8 went for more than the "average" value of an IP address. Larger contiguous blocks are more useful, and having a /8 has got to have some prestige value as well.

lucb1e 6y ago
But on the other hand, wholesale items are rarely sold at piece prices. I'm not sure the price increased that much just by having a large block. Are there other big blocks sold for known prices?
- bgpdude 6y ago
  the only recent one i'm aware of is this one: http://www.southgatearc.org/news/2020/october/sale-of-amateur-radio-amprnet-tcp-ip-addresses.htm http://www.southgatearc.org/news/2020/october/sale-of-amateu... AWS paid $108 million for this /10. That’s $25.74 per IP address.
perlgeek 6y ago
I've talked to our resident backbone engineer about that about a year ago, and he said that larger blocks transfer for a lower value per IP, on average. I was surprised by that, but he showed me data that supported that. Update: I just found http://www.circleid.com/posts/20180731_the_ipv4_market_2018_mid_year_report/ http://www.circleid.com/posts/20180731_the_ipv4_market_2018_... which says > As we head into the last half of 2018, small blocks are selling for just over $18/number, mid-blocks are in the $15-$18/number range, and large blocks have surpassed the $20/number threshold so it seems there's a bit of an U shape, where mid-sized blocks have the lowest price per address.
- toast0 6y ago
  I think this makes sense. A lot of networks need a few addresses, so /24 - /22 is probably pretty active, and you have to justify the addresses, so someone who really only needs a /24 isn't able to buy a /21 and make it work; very few people need huge swaths, but those will be able to really use them and may value having them contiguous. In the middle, it doesn't make much difference having a /16 or two /17s or four /18s, so there's a lot more flexibility, and you take the lowest price per IP if there's a block you can use.

beckler 6y ago

I was working at GE when AWS bought GE's 3.0.0.0/8 block, and it caused a massive headache. GE's network is gigantic, and there are a ridiculous number of assets. They didn't give much notice about the sale, so they shimmed all of the routing internally, but lots of services still pointed to the block. There supposedly was an agreement that Amazon wouldn't start provisioning them for a certain amount of time, but whatever amount of time was specified, they either didn't honor it, or it wasn't enough time. We were also moving assets over to AWS, and all of these things going on simultaneously caused what we called the three-pocalypse. We would occasionally run across issues with external sites or newly provisioned lambdas who were on Amazon's new 3.0.0.0/8 block, but we couldn't reach them because internally that IP address didn't exist. At the same time, they would open up a small block to allow access to those external sites, and then some internal service would no longer respond. Repeat ad nauseam. It was also compounded by the fact that there are countless teams in GE and not everyone would connect with who made what changes.

RKearney 6y ago
Sounds like the perfect time to migrate to IPv6!
- sroussey 6y ago
  IP6 on AWS ECS is a pain, and impossible if you use their recommended and dedicated command line tool.
  - Ericson2314 6y ago
    This has always seemed deeply suspicious to me. It's like AWS is trying to prop up the value of its speculation on IPv4 addresses.
    - leetcrew 6y ago
      or merely lazy. why spend time/$$$ supporting IPv6 when you have plenty of IPv4 addresses?

ryan_j_naughton 6y ago

Can someone explain why these have such value? What systems cannot handle IPV6 yet?

casperb 6y ago
Like 60-70% of all internet users: https://www.google.com/intl/en/ipv6/statistics.html#tab=ipv6-adoption&tab=ipv6-adoption https://www.google.com/intl/en/ipv6/statistics.html#tab=ipv6...
- Spivak 6y ago
  This is a little misleading. This just measures users that are using IPv6 not users that would be able to use IPv6 if IPv4 stopped working. My local network and ISP are perfectly capable of using IPv6 but you have to call them to switch.
- jcranmer 6y ago
  Zoom into 2020, and you can clearly see when the lockdowns due to the pandemic start on that chart. The other interesting artifact is that it looks like IPv6 network rollout tends to almost exclusively happen in the April-July timeframe.
gertrunde 6y ago
Imagine that you have two ways of addressing your system. 1) Allows you to access almost 100% of your potential customers. 2) Allows you to access around 35% of your potential customers. Which one would you choose any why?
- ArchOversight 6y ago
  Both. Especially if that one with 35% of my customers provides me with lower latency, higher throughput and costs me less in CPU time/power to run my traffic across. The rest of the people I need to eat the cost for...
  - 6y ago

vmception 6y ago

The addressing apocalypse that never comes

remus 6y ago
Presumably there will come a point where the cost of purchasing enough ipv4 addresses outweighs the cost of making a project work with ipv6. It'll be an interesting piece of economics to see where that boundary is.
- wmf 6y ago
  The economics (for consumer ISPs) are explored at http://www.asgard.org/documents.html http://www.asgard.org/documents.html
jedberg 6y ago
Carrier grade NAT and cell phones being IPv6 only helped stem the tide a lot.
- skipnup 6y ago
  And my cell phone connection in Germany is IPv4 only :-(
  - usr1106 6y ago
    Well, Germany, they don't even have cell phone coverage in every village. And where they have it data rates can be abysmal. Yesterday the news hit the mainstream media that Nokia got a NASA deal to build 4G on the moon. Couldn't resist thinking building it in Germany would be more useful. Disclaimer: I lived in the county over 30 years and still have strong links.
- eznzt 6y ago
  I don't think "cell phones being IPv6 only" is a widespread phenomenon. If anything, "cell phones being CGNAT only" is the most popular case.
6y ago

mmm_grayons 6y ago

It's interesting when you look at it from this perspective. AWS and other big IPv4 owners have a financial incentive to hold back IPv6 deployment in a convenient and scalable fashion, because that would quickly wipe out billions in value of a scarce resource. It's like some people who own NFA firearms: they support keeping the regulations because a lightning link isn't worth $15k without it, it's just a cheap piece of stamped sheet metal. Edit: it's also an easy way to outcompete others; IPv4 space has become a huge capex for starting a cloud provider.

bluGill 6y ago
Until one runs out of IPv4 addresses. I think it is safe to say that Amazon, Facebook, and google each have data centers big enough that IPv4 address management is a headache. I believe Facebook is internally IPv6 (that is server to server communications - their public space obviously has IPv4, and their desktop space can get by with NAT and IPv4 if they want to). Google has done enough promoting of IPv6 that I think they are the same. I'm not sure what AWS is - they sell server time so they have more need for IPv4 to the servers (google cloud has the same concerns)
- ArchOversight 6y ago
  Facebook only has IPv4 at the edge. Everything internal is IPv6.
  - samoa42 6y ago
    suggests that ipv6 is here to replace the rfc1918 space but "the internet" largely remains ipv4. the high v6 penetration in mobile networks likewise suggests that smartphones are generally not a part of the network as much as dedicated leaf nodes that require no inbound connections.
    - p1mrx 6y ago
      There's a difference between "IPv4 is only present at the edge" and "only IPv4 is present at the edge". I think Facebook is doing IPv4+IPv6 at the edge, and IPv6-only internally.

iJohnDoe 6y ago

It’s a good future-proofing investment by AWS. IPv4 is not going away anytime soon. We’ll be dependent on IPv4 for a long time. IPv4 scarcity only hits in certain segments. It doesn’t hit the major players. The real problem is the hoarding by the major players. AWS will most likely never use that many IP addresses. I know that sounds like a stretch based on what we’re led to believe about running out of IPv4 addresses. I don’t know their IPv4 utilization, so it’s a guess. My guess is based on how we deploy internet facing services these days. It’s less dependent on needing lots of public IPs.

labawi 6y ago
IMO it's good for the internet if few entities were to hoard most IPv4 addresses. Only while IPv4's are sufficiently available will we keep using IPv4-only networks and keep prolonging our miseries. My ISP only gives CGNAT or is happy to rent IPv4 for a premium, one address at a time. Keeping IPv6 out helps the IPv4 margins and staves off competition.
shiftpgdn 6y ago
According to the article AWS usage is at 47% of their IP pool.
kaliszad 6y ago
You cannot use IPv4 100% efficiently. Even if you approach 90% usage, that would probably be very optimistic. AWS has probably millions of physical servers by now (they had ca. 1,4 million servers by December 2014 it seems) each with probably hundreds of services each needing an IP. Yes, you can stretch 10.0.0.0/8 with NATs all over the place but at some point, you need the public IP. With that kind of appetite you need multiple /8s of IPv4 space just to keep things running. I don't think they are crazy buying that much address space and concurrently improving IPv6 support - they most likely really need it.
- tudelo 6y ago
  > You cannot use IPv4 100% efficiently. Can you expand on this a bit? I'm not a networking guru but it's my understanding that an IP address is an IP address, how could it be used more or less efficient?

Ecco 6y ago

The sad thing is that's probably why IPv6 is never going to be a thing. Why would all major tech companies be willing to write off such expensive assets?

lolc 6y ago
They can try and resist. But they will lose. For example: I've set up a IP6-only backend because it was easier than getting IP4-connectivity. And when I tried Travis and they couldn't connect to that backend, I dropped Travis immediately. That is still uncommon, but I'm sure I'm not the only one who's made a decision in that direction.
tptacek 6y ago
Because they routinely do? They're tech companies. Everything they own depreciates.
deleted 6y ago
[deleted]

valenciarose 6y ago

The artificial resource constraints created by IPv4 are a substantial barrier to entry. Prediction: we’re going to see a court case that forces a mandatory retirement of IPv4 sometime in the next 15 years.

lolc 6y ago
I think that IP4 will see itself out when service degradation becomes generally noticeable. CGNAT is really painful to operate. Remember the IE6 banners? There will be a time in ten years (haha it's always ten years!) when you'll see "Our site runs with degraded performance over IPv4. Please contact your administrator."
- anderspitman 6y ago
  > CGNAT is really painful to operate It also makes it much more difficult for customers to stress your upload bandwidth though.
  - lolc 6y ago
    You mean to say customers get a degraded experience on CGNAT.
    - anderspitman 6y ago
      Yup. If you can't host a webserver from your home IP, that's one less thing your ISP needs to worry about taxing their network. It's sad the incentives just aren't aligned for a decentralized internet.
- eznzt 6y ago
  >CGNAT is really painful to operate. For whom? There are lots of providers (for example Huawei) that offer turnkey solutions for ISPs to painlessly roll out CGNAT.
  - lolc 6y ago

varenc 6y ago

MIT used to have all of 18.0.0.0/8 but they sold half of it to AWS a few years ago. Because MIT had so many IP addresses they didn't need to use a NAT. Everyone connecting over wifi got a real world 18.* IP address. It also made network architecture easy. Every MIT building got its own B class subnet. Hilariously, even the MIT boat house which couldn't have had more than a dozen computers online at once had 65,534 addresses. It made it very easy to find out which building someone was in based on their IP.

deleted 6y ago
[deleted]
angulardragon03 6y ago
My old uni still has two /16s. Everyone on campus gets their own “public” IP (behind the firewall, of course).
icedchai 6y ago
IP blocks were handed out like candy in the mid-90's and earlier. I have a /24 ("class C") personally, and know of several others who do also.
stordoff 6y ago
My old university has three /16s, and a bunch of other IP blocks, so gave public IPs to everyone (though they now seem to be using private IPs for some purposes). I never looked at how the public machines were allocated, but it did make picking out traffic from the residences trivial - they reverse DNS resolved to <userid>.<college.>cam.ac.uk

unklefolk 6y ago

What does the slash mean in the article? Where it says “... in 2018 GE sells 3.0.0.0/8” what does the “/8” mean? I guess it indicates the range of addresses?

wging 6y ago
Yep. https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing#CIDR_notation https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing... for more info.
codegladiator 6y ago
https://www.calculator.net/ip-subnet-calculator.html https://www.calculator.net/ip-subnet-calculator.html Checkout the table on the page (scroll down)
deleted 6y ago
[deleted]
SheepSlapper 6y ago
You're right that it indicates a range of addresses, the number after the slash is the number of starting bits that the addresses all share. Check out CIDR Blocks for a more complete answer: https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing#CIDR_blocks https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing...
justusthane 6y ago
/8 is the same as a 255.0.0.0 subnet mask. Also known as a class A subnet. In other words, literally any IP address that starts with “3.” In other other words, 16,777,216 IP addresses. In other other other words, about half a percent of the total number of public IPv4 addresses. Crazy.

mbreese 6y ago

Id love to know similar stats on IPv4 space for Google Cloud and Azure.

neiman 6y ago

If we insist on IPv4, at least treat those IPs as a public resource of humanity, and don't just let anyone horde them in all kinds of strange deals.

wmf 6y ago
Officially, hoarding has never been allowed. You can't buy IPs unless you have a plan to use them. And there's nothing strange about a company with tens of millions of VMs buying tens of millions of addresses.
- neiman 6y ago
  People hoarded toilet paper when Corona began with full intention to use them. People hoard food in wartime with intention to eat it. So yes, Amazon hoard IPv4 with intention to maybe use them in the future. But hording it is, and in a non-hoard system they'd rent - not buy - IPv4 with a month notice or so.

anderspitman 6y ago

I've toyed with the idea of buying a /24 as a (probably terrible) investment that I would actually get at least some use out of in the short term (some VPSes like vultr let you bring your own IPs). Anyone have enough experience with this to talk me out of it?

Thorentis 6y ago
With my investing luck, the world would suddenly switch to 100% IPv6 the day after the sale went through.
- mikestew 6y ago
  If that's all it takes, I'm willing to start the co-op that fronts you the money.
bibabaloo 6y ago
In a similar vein, is there a cost effective way to purchase a single ipv4 address? Just for the novelty I think it'd be cool if I could spend ~$100 to have an ipv4 address that's permanently mine.
- jlgaddis 6y ago
  I've got plenty I'm not using, I'll give you one. Here, this one's an easy one to remember, you can have it: 127.1. Enjoy!
- fanf2 6y ago
  The minimum IPv4 allocation is a /24 because that is the smallest network that can be globally routed. Smaller networks are filtered out of BGP to stop routing tables getting too big too fast. Anything smaller than /24 has to be rented from a network provider who will advertise an aggregate route covering many customers.
pera 6y ago
AFAIK you need to demonstrate a real need to actually acquire a block, this is not like buying a domain name.