3 ms·

None of my comments are TCP specific, but as I said - at 3 PoPs, you won't see routing weirdness; announcing the same prefix from 70 PoPs will cause the likelih

None of my comments are TCP specific, but as I said - at 3 PoPs, you won't see routing weirdness; announcing the same prefix from 70 PoPs will cause the likelihood of route instability to increase. For the case of UDP, hopefully no one will notice, as your lost packets will just go to the next available route; for TCP connections, a route withdrawal sending you to the next nearest PoP will lead to a broken connection and clients seeing badness in their browsers. Anycast is an IPv4 quirk in that it wasn't designed; it "just works" because of the BGP route decision process. Yes, you can announce your prefix out of multiple PoPs, but how people choose to use the routes you announce is where things get "quirky". In theory there's no difference between announcing a /24 IPv4 and asking us all to accept it, and doing the same with a /48, but in practice you'll burn an unacceptably large amount of IP space, IMHO, and if you tell ARIN that you need a second /48 block because you burned your first with less than 25% utilization, they simply won't let you have it. Disagreed on your last point. The abundance of prefixes isn't the problem IMHO; it's the acceptance of route announcements from upstreams. If upstreams were to start accepting and passing along more granular routes, maybe we'll start seeing more anycast deployments, but I doubt it.

clinta 11y ago
Agreed that if doing TCP, you either need to limit the PoPs, or be doing it with a an app that is highly resilient and capable of dealing with unexpected loss of a session and reconnections. The same is true of UDP. For stateless short burst things like DNS and NTP it's not a big deal, but it is for video conferencing and other stateful protocols built on top of UDP. It's not a layer 4 issue as much as an application issue. As to the second point, ARIN could not possibly require 25% utilization. That's 3*10^23 addresses on a /48. Or if they're counting utilization as the number of subnets, that 16,384 /64 networks. Look at some sample companies, Valve has a /44 from ARIN. They're only even announcing 6 /48 prefixes right now. Netflix has a /32, they're announcing less than 20 /48 prefixes. I worked at a small MSP who provides ISP services to less than 50 clients, and they were allocated a /32 years ago and have yet to announce a single prefix. Burning a /48 for an anycast network is not a big deal, and getting another /48 is trivial. Getting a /24 for an IPv4 anycast network is much harder. So yes, for IPv4 anycast to take off we need the ability to advertise longer prefixes, but /48's are abundant enough that one can be sacrificed.

clinta 11y ago

Agreed that if doing TCP, you either need to limit the PoPs, or be doing it with a an app that is highly resilient and capable of dealing with unexpected loss of a session and reconnections. The same is true of UDP. For stateless short burst things like DNS and NTP it's not a big deal, but it is for video conferencing and other stateful protocols built on top of UDP. It's not a layer 4 issue as much as an application issue. As to the second point, ARIN could not possibly require 25% utilization. That's 3*10^23 addresses on a /48. Or if they're counting utilization as the number of subnets, that 16,384 /64 networks. Look at some sample companies, Valve has a /44 from ARIN. They're only even announcing 6 /48 prefixes right now. Netflix has a /32, they're announcing less than 20 /48 prefixes. I worked at a small MSP who provides ISP services to less than 50 clients, and they were allocated a /32 years ago and have yet to announce a single prefix. Burning a /48 for an anycast network is not a big deal, and getting another /48 is trivial. Getting a /24 for an IPv4 anycast network is much harder. So yes, for IPv4 anycast to take off we need the ability to advertise longer prefixes, but /48's are abundant enough that one can be sacrificed.