5 ms·

It's nice having cheap/free DNS from people like Rackspace and Amazon, but situations like these make you realise that it's sensible to use a company like Dyn (

It's nice having cheap/free DNS from people like Rackspace and Amazon, but situations like these make you realise that it's sensible to use a company like Dyn (http://dyn.com/ http://dyn.com/) that are experts in highly-available DNS, rather than something that's a small part of a hosting provider's services. It's easy to forget that you can have redundancy in your load balancers, web servers and databases (replication, multiple data centres, etc), but DNS is how you're found by the rest of the Internet. No DNS resolution = no one reaches your expensive, lovingly-crafted infrastructure.

blfr 12y ago
On the other hand, the larger your network, the more capacity you have for dealing with attacks.
mikegioia 12y ago
I think Rackspace invests a lot of money into their DNS infrastructure. Yea it's free but its mostly because you're paying so much extra for the servers and support. No DNS doesn't always mean no one reaches your expensive infrastructure. That may be true for websites relying on lots of random traffic, but most if not all of our customers have been to our site before so there's a strong chance that the DNS has already been cached on their computer or router.
- Khao 12y ago
  But that's no use if you use the default TTL of 2-3 hours that I always see whenever I configure new dns entries. Sensible TTL should be at least 24-48 hours, maybe even more for your returning users to not be affected by those kind of outages.
  - dangrossman 12y ago
    A 24-48 hour TTL means you're hosed for 24-48 hours if your hosting service has an extended outage because you can't point any of those visitors at the IP addreses at another host. With a short TTL on the other hand, the worst-case scenario is that your DNS provider is offline and you have to change nameservers at the registrar. With a hosting outage, you can just point the domains at the new IPs and be back up in minutes to hours.
    - larrys 12y ago
      Agree with your points. "you have to change nameservers at the registrar" Would add that that this is also one of the reasons not to use your registrar for DNS, as a generality.
grinich 12y ago
Has Route 53 actually had issues in the past? It seems like a failure there would also take down most AWS services.
- jbinto 12y ago
  I don't believe it was a DDoS, but Route 53 stopped resolving records for Cloudfront distributions for a few hours on Nov 26 2014. This affected everyone who uses CloudFront (which is a lot), including Amazon.com itself. There, all product images were timing out. https://news.ycombinator.com/item?id=8665367 https://news.ycombinator.com/item?id=8665367 http://www.forbes.com/sites/benkepes/2014/11/26/in-response-to-azure-outages-amazon-has-its-day-of-doom-aws-cloudfront-suffers-global-issue/ http://www.forbes.com/sites/benkepes/2014/11/26/in-response-...
drzaiusapelord 12y ago
Four regions have "emergenecy maintanance" right now at Dyn. Of course, we don't know whats the cause of that. I imagine if its also a DDOS they'll probably not reveal that. I have heard from other sysadmins that they've had problems with them as well. I have DNSmadeeasy and they seem to do okay, but only because they're such a small player and avoid being targetted too often. Sorry, but there's no silver bullet here. Cloud providers get hacked and DDOS'd all the time. Kiddies and morons find it amusing to do so. Roll your own if you want to avoid being attached to such a big target and have a cloud provider just be your secondary namesever.
larrys 12y ago
We've used (and still use) dyn for some things. What they do unfortunately is charge you for the queries but there is no way of knowing which host has exceeded the limits query wise. When we first started using them they stated that even though we were exceeding the limits according to how they measure "not to worry you won't get charged". Of course perhaps 1 to 1.5 years later they now do charge (email out of the blue one day) and constantly try to bump us to a higher level service. [1] All this for a few zones that use to run comfortably on a 2 servers that we had many other things going on as well as DNS. (Now they claim we get 2.5 million queries per month but there is no way to determine exactly if that is true or for what host, ie foo.domain.com vs. www.domain.com is causing the excess queries). [1] Which is more than paying the overage charges which is what we do every month.
troydavis 12y ago
The solution isn't to change which single point of failure - that is, DDoS target - you depend on, it's to depend on more than one network. We recently went through this with another provider and wrote about it (http://blog.papertrailapp.com/dns-outage-on-monday-december-1-2014/); http://blog.papertrailapp.com/dns-outage-on-monday-december-... the gist: "Relying on one DNS infrastructure, no matter how large or distributed, is an unnecessary risk"

blfr 12y ago

On the other hand, the larger your network, the more capacity you have for dealing with attacks.

mikegioia 12y ago

I think Rackspace invests a lot of money into their DNS infrastructure. Yea it's free but its mostly because you're paying so much extra for the servers and support. No DNS doesn't always mean no one reaches your expensive infrastructure. That may be true for websites relying on lots of random traffic, but most if not all of our customers have been to our site before so there's a strong chance that the DNS has already been cached on their computer or router.

Khao 12y ago
But that's no use if you use the default TTL of 2-3 hours that I always see whenever I configure new dns entries. Sensible TTL should be at least 24-48 hours, maybe even more for your returning users to not be affected by those kind of outages.
- dangrossman 12y ago
  A 24-48 hour TTL means you're hosed for 24-48 hours if your hosting service has an extended outage because you can't point any of those visitors at the IP addreses at another host. With a short TTL on the other hand, the worst-case scenario is that your DNS provider is offline and you have to change nameservers at the registrar. With a hosting outage, you can just point the domains at the new IPs and be back up in minutes to hours.
  - larrys 12y ago
    Agree with your points. "you have to change nameservers at the registrar" Would add that that this is also one of the reasons not to use your registrar for DNS, as a generality.

grinich 12y ago

Has Route 53 actually had issues in the past? It seems like a failure there would also take down most AWS services.

jbinto 12y ago
I don't believe it was a DDoS, but Route 53 stopped resolving records for Cloudfront distributions for a few hours on Nov 26 2014. This affected everyone who uses CloudFront (which is a lot), including Amazon.com itself. There, all product images were timing out. https://news.ycombinator.com/item?id=8665367 https://news.ycombinator.com/item?id=8665367 http://www.forbes.com/sites/benkepes/2014/11/26/in-response-to-azure-outages-amazon-has-its-day-of-doom-aws-cloudfront-suffers-global-issue/ http://www.forbes.com/sites/benkepes/2014/11/26/in-response-...

drzaiusapelord 12y ago

Four regions have "emergenecy maintanance" right now at Dyn. Of course, we don't know whats the cause of that. I imagine if its also a DDOS they'll probably not reveal that. I have heard from other sysadmins that they've had problems with them as well. I have DNSmadeeasy and they seem to do okay, but only because they're such a small player and avoid being targetted too often. Sorry, but there's no silver bullet here. Cloud providers get hacked and DDOS'd all the time. Kiddies and morons find it amusing to do so. Roll your own if you want to avoid being attached to such a big target and have a cloud provider just be your secondary namesever.

larrys 12y ago

We've used (and still use) dyn for some things. What they do unfortunately is charge you for the queries but there is no way of knowing which host has exceeded the limits query wise. When we first started using them they stated that even though we were exceeding the limits according to how they measure "not to worry you won't get charged". Of course perhaps 1 to 1.5 years later they now do charge (email out of the blue one day) and constantly try to bump us to a higher level service. [1] All this for a few zones that use to run comfortably on a 2 servers that we had many other things going on as well as DNS. (Now they claim we get 2.5 million queries per month but there is no way to determine exactly if that is true or for what host, ie foo.domain.com vs. www.domain.com is causing the excess queries). [1] Which is more than paying the overage charges which is what we do every month.

troydavis 12y ago

The solution isn't to change which single point of failure - that is, DDoS target - you depend on, it's to depend on more than one network. We recently went through this with another provider and wrote about it (http://blog.papertrailapp.com/dns-outage-on-monday-december-1-2014/); http://blog.papertrailapp.com/dns-outage-on-monday-december-... the gist: "Relying on one DNS infrastructure, no matter how large or distributed, is an unnecessary risk"