3 ms·

Author here. We do actually offer all of these options. We offer a public IP address for A records pointing to a our main load balancer. This will send all tra

by bobfunk 10y ago

Author here. We do actually offer all of these options. We offer a public IP address for A records pointing to a our main load balancer. This will send all traffic to a single origin instead of serving your HTML pages out of our global CDN. We also offer DNS hosting for pro plans and up. When you move your DNS to Netlify, the caveat about naked domains doesn't apply (as mentioned in the first paragraph), since we hook the domain record straight into our global traffic director. For enterprise customers we also offer an anycasted IP address that lets you use our CDN with a normal A record, but we still recommend either using our DNS hosting or a www domain since the DNS based traffic direction is faster at responding to localized issues and offers more precise traffic distribution.

jsjohnst 10y ago
Wouldn't a simpler (for the end customer, not for you) solution be to use Anycast on a (or block of) IP addresses and then let folks always use A records as intended? Solves the ANAME non-local caching issue and also handles people using DNS servers not nearby to them.
- bobfunk 10y ago
  We do run an anycast CDN network, but there's a lot of limitations on BGP routing compared to CDN based traffic direction. We can only route BGP requests to hardware we control, whereas we can add PoPs in all the major cloud providers on our DNS based network. We can then use tools like Cedexis or DYNs internet intelligence to identify where the different cloud providers have the best networking and peering agreements and piggy back on that + their DDoS mitigation. This means we get a combination of all the best AWS/Google Cloud/Rackspace/DO, etc, etc has to offer in that aspect. On the DNS based traffic director we can also do very quick traffic decisions (20s TTL, instant changes) whereas on our BGP routed anycast IP we have to be more conservative and force 10 minute intervals between any up/down changes for a PoP.
  - kyledrake 10y ago
    I did GeoDNS + Unicast IPs for a while. I had a really rough time making it work, and we ended up building our own anycast network (https://status.neocities.org https://status.neocities.org) Aside from the root domain issues (and less options for market-priced bandwidth), "GeoDNS + Cloud" pushes your traffic into someone else's ASN, which means complaints end up being sent to them, and your hosting is effectively governed not just by one, but by two different ToSes. This isn't a big deal for a couple thousand sites (unless they're huge), but once you start getting into the hundreds of thousands, you'll see a significant spike in issues (phishing, malware, spam, DMCA, legal threats, etc.) that get sent to whomever owns that IP address. After getting too many of these complaints, those other providers can decide you're just not worth the effort and boot you off their servers. Crazy hypothesis? Sounds like it would be, but it happens: https://twitter.com/surge_sh/status/685164708861624325 https://twitter.com/surge_sh/status/685164708861624325. DO did the same thing to us when we tried to use them for part of our CDN early on. After that, I tried three other cloud services that either did the same thing or threatened to do the same thing (to say nothing about the ridiculously overpriced bandwidth). The choice we were left with: Get our own AS, or die. Mind you, this was over < 30 abuse reports per month, not thousands. Most of these providers are designed for a single company or a wordpress blog, they're not designed (and not really equipped) for usage as infrastructure for a web hosting provider with hundreds of thousands (or millions) of customers. Building out the anycast CDN was a "drinking from the firehose" experience and had some upfront costs I would have rather not paid, but it solved this existential problem for us permanently, and probably saved our life. From experience, I do think you'll have to do this eventually (or at least do GeoDNS + unicast with your own IPs and AS).
    - zaroth 10y ago
      Have you written up your experience with building out the anycast CDN? That would be extremely interesting!
      - jsjohnst 10y ago
        I'd be interested in reading that too
iaml 10y ago
You might want to fix the "two benefits: 1. [...] 1." in the article.
waawal 10y ago
Does this anycasted ip actually serve the html page/assets or does it reply with a redirect to a "stable" ip announced from the same pop for actually serving the assets?
- bobfunk 10y ago
  It serves the HTML or assets directly
  - ec109685 10y ago
    Don't you have to worry about the route changing during a TCP connection, causing the destination physical server to change, severing the connection?
    - waawal 10y ago
      Yes, that is also my concern. I hope Matt will share his experience here.

by bobfunk 10y ago

jsjohnst 10y ago

Wouldn't a simpler (for the end customer, not for you) solution be to use Anycast on a (or block of) IP addresses and then let folks always use A records as intended? Solves the ANAME non-local caching issue and also handles people using DNS servers not nearby to them.

bobfunk 10y ago
We do run an anycast CDN network, but there's a lot of limitations on BGP routing compared to CDN based traffic direction. We can only route BGP requests to hardware we control, whereas we can add PoPs in all the major cloud providers on our DNS based network. We can then use tools like Cedexis or DYNs internet intelligence to identify where the different cloud providers have the best networking and peering agreements and piggy back on that + their DDoS mitigation. This means we get a combination of all the best AWS/Google Cloud/Rackspace/DO, etc, etc has to offer in that aspect. On the DNS based traffic director we can also do very quick traffic decisions (20s TTL, instant changes) whereas on our BGP routed anycast IP we have to be more conservative and force 10 minute intervals between any up/down changes for a PoP.
- kyledrake 10y ago
  I did GeoDNS + Unicast IPs for a while. I had a really rough time making it work, and we ended up building our own anycast network (https://status.neocities.org https://status.neocities.org) Aside from the root domain issues (and less options for market-priced bandwidth), "GeoDNS + Cloud" pushes your traffic into someone else's ASN, which means complaints end up being sent to them, and your hosting is effectively governed not just by one, but by two different ToSes. This isn't a big deal for a couple thousand sites (unless they're huge), but once you start getting into the hundreds of thousands, you'll see a significant spike in issues (phishing, malware, spam, DMCA, legal threats, etc.) that get sent to whomever owns that IP address. After getting too many of these complaints, those other providers can decide you're just not worth the effort and boot you off their servers. Crazy hypothesis? Sounds like it would be, but it happens: https://twitter.com/surge_sh/status/685164708861624325 https://twitter.com/surge_sh/status/685164708861624325. DO did the same thing to us when we tried to use them for part of our CDN early on. After that, I tried three other cloud services that either did the same thing or threatened to do the same thing (to say nothing about the ridiculously overpriced bandwidth). The choice we were left with: Get our own AS, or die. Mind you, this was over < 30 abuse reports per month, not thousands. Most of these providers are designed for a single company or a wordpress blog, they're not designed (and not really equipped) for usage as infrastructure for a web hosting provider with hundreds of thousands (or millions) of customers. Building out the anycast CDN was a "drinking from the firehose" experience and had some upfront costs I would have rather not paid, but it solved this existential problem for us permanently, and probably saved our life. From experience, I do think you'll have to do this eventually (or at least do GeoDNS + unicast with your own IPs and AS).
  - zaroth 10y ago
    Have you written up your experience with building out the anycast CDN? That would be extremely interesting!
    - jsjohnst 10y ago
      I'd be interested in reading that too

iaml 10y ago

You might want to fix the "two benefits: 1. [...] 1." in the article.

waawal 10y ago

Does this anycasted ip actually serve the html page/assets or does it reply with a redirect to a "stable" ip announced from the same pop for actually serving the assets?

bobfunk 10y ago
It serves the HTML or assets directly
- ec109685 10y ago
  Don't you have to worry about the route changing during a TCP connection, causing the destination physical server to change, severing the connection?
  - waawal 10y ago
    Yes, that is also my concern. I hope Matt will share his experience here.