22 ms·

The problem of running TCP over Anycast addressing tends to be overblown. Cloudflare, for example, does all of their client-facing HTTP traffic over Anycast and

The problem of running TCP over Anycast addressing tends to be overblown. Cloudflare, for example, does all of their client-facing HTTP traffic over Anycast and so you can consider that as existence proof for it being a viable strategy. Routes simply don't seem to shift all that frequently in practice.

akvadrako 8y ago
I used to operate an anycast DNS service and we monitored it from probes all over the world. Sometimes a random ISP in USA/Europe would start sending queries to Japan and that could last for seconds to hours. I'm not sure how often it happens, but it will certainly be enough that it will cause broken connections for a few users. I'm not sure how Cloudflare deals with it - potentially they could share the TCP connection data between datacenters when needed. Or maybe it has something to do with their how edgy their setup is - if it's mostly intra-ISP cache servers being anycasted, the routing should be more stable.
- Artemis2 8y ago
  AFAIK a lot of Google’s HTTPS traffic is also routed using anycast. They also offer this to GCP customers under GCLB (https://cloud.google.com/load-balancing/ https://cloud.google.com/load-balancing/). I don’t know how they deal with changes in routes.
lumisota 8y ago
Some interesting anycast stability measurements: http://tma.ifip.org/wordpress/wp-content/uploads/2017/06/tma2017_paper39.pdf http://tma.ifip.org/wordpress/wp-content/uploads/2017/06/tma...
zzzcpan 8y ago
Anycast still has enough problems, that it's a miracle some are using it more or less successfully. It's not like pure DNS-based routing, where you can throw a bunch of cheap dedicated or virtual servers at it in various countries, measure dns/http latencies in the browser and then just choose lowest latency nodes for IPs, subnets.

22 ms·

The problem of running TCP over Anycast addressing tends to be overblown. Cloudflare, for example, does all of their client-facing HTTP traffic over Anycast and

by markonen 8y ago

akvadrako 8y ago
I used to operate an anycast DNS service and we monitored it from probes all over the world. Sometimes a random ISP in USA/Europe would start sending queries to Japan and that could last for seconds to hours. I'm not sure how often it happens, but it will certainly be enough that it will cause broken connections for a few users. I'm not sure how Cloudflare deals with it - potentially they could share the TCP connection data between datacenters when needed. Or maybe it has something to do with their how edgy their setup is - if it's mostly intra-ISP cache servers being anycasted, the routing should be more stable.
- Artemis2 8y ago
  AFAIK a lot of Google’s HTTPS traffic is also routed using anycast. They also offer this to GCP customers under GCLB (https://cloud.google.com/load-balancing/ https://cloud.google.com/load-balancing/). I don’t know how they deal with changes in routes.
lumisota 8y ago
Some interesting anycast stability measurements: http://tma.ifip.org/wordpress/wp-content/uploads/2017/06/tma2017_paper39.pdf http://tma.ifip.org/wordpress/wp-content/uploads/2017/06/tma...
zzzcpan 8y ago
Anycast still has enough problems, that it's a miracle some are using it more or less successfully. It's not like pure DNS-based routing, where you can throw a bunch of cheap dedicated or virtual servers at it in various countries, measure dns/http latencies in the browser and then just choose lowest latency nodes for IPs, subnets.