4 ms·

Mirror your data onto another provider continuously (log shipping/rsync), Switch DNS. Ansible works for this stuff as it allows you to can the task of “quick g

by cjsuk 9y ago

Mirror your data onto another provider continuously (log shipping/rsync), Switch DNS. Ansible works for this stuff as it allows you to can the task of “quick get me a production environment up on Linode!” If you can afford some downtime you don’t need a hot standby just roll out everything into new provider and you’re done. I’ve done this on very large scale environments and small ones and it’s achievable for even small organisations. The killer is avoid anything you can’t run on bare metal servers.

therealmarv 9y ago
I also do this approach. Get a decent, from your infrastructure independent DNS provider and take care of your Ansible scripts. This way in emergency you do a one liner, have a new production server running and change DNS settings.
- mschuster91 9y ago
  In addition I'd suggest getting a second domain in a TLD operated by another company in another country than the primary TLD is, and teaching your customers/users that both are valid. This protects you from three things: 1) your DNS provider having issues (even Route53 sometimes has them, https://mwork.io/2017/03/14/aws-route53-dns-outage-impacts-last-almost-a-full-day/ https://mwork.io/2017/03/14/aws-route53-dns-outage-impacts-l...) 2) legal issues, when one of your domains gets seized or the provider gets pressured into cancelling, just as has happened with Pirate bay, SciHub and friends, gambling or sites with user generated content that may be illegal or frowned upon in some countries or for the latter to the Nazi site Daily Stormer (although I'm glad for it being down, it's a perfect example what can happen in a very short time frame) 3) (edit, after suggestion below) the entire TLD going down because the TLD DNS provider has issues, which also happens from time to time.
  - cjsuk 9y ago
    Good advice. We had some serious trouble when .io went down a few weeks back.

by cjsuk 9y ago

therealmarv 9y ago

I also do this approach. Get a decent, from your infrastructure independent DNS provider and take care of your Ansible scripts. This way in emergency you do a one liner, have a new production server running and change DNS settings.

mschuster91 9y ago
In addition I'd suggest getting a second domain in a TLD operated by another company in another country than the primary TLD is, and teaching your customers/users that both are valid. This protects you from three things: 1) your DNS provider having issues (even Route53 sometimes has them, https://mwork.io/2017/03/14/aws-route53-dns-outage-impacts-last-almost-a-full-day/ https://mwork.io/2017/03/14/aws-route53-dns-outage-impacts-l...) 2) legal issues, when one of your domains gets seized or the provider gets pressured into cancelling, just as has happened with Pirate bay, SciHub and friends, gambling or sites with user generated content that may be illegal or frowned upon in some countries or for the latter to the Nazi site Daily Stormer (although I'm glad for it being down, it's a perfect example what can happen in a very short time frame) 3) (edit, after suggestion below) the entire TLD going down because the TLD DNS provider has issues, which also happens from time to time.
- cjsuk 9y ago
  Good advice. We had some serious trouble when .io went down a few weeks back.