4 ms·

Remember guys & gals: this can happen to you. I am a fanatical Linode customer, and they are, in general, an extremely rock solid service, but that doesn't alle

Remember guys & gals: this can happen to you. I am a fanatical Linode customer, and they are, in general, an extremely rock solid service, but that doesn't alleviate my responsibility to ensure my application is available regardless of datacenter outage. Power outages happen to EVERY datacenter. There are some very well documented and understood ways to alleviate this problem.

KaeseEs 16y ago
Care to enumerate a few of them?
- al_james 16y ago
  More than one server across multiple datacenters is the only way. The use either a load balancer (which brings its own problems) or some kind of status aware DNS scheme to route traffic to live servers. I wish linode would offer some kind of high availability load balancing like amazon ec2 does, that really is a killer feature.
  - rbranson 16y ago
    I wouldn't be so sure that ELB's are actually failing whole IP addresses across availability zones. Given the current ELB architecture is heavily dependent on dynamic DNS resolution, I am pretty suspicious that they are using DNS for failover. The separate set of DNS servers specifically for elb.amazonaws.com and the 60 second TTLs on the CNAMEs they generate seem to indicate this. Of course, there is no documentation to either confirm, or deny this. Amazon simply states in vague terms that it's highly available, but doesn't state how it's designed, or even possible failure outcomes.
    - al_james 16y ago
      Hmmm. Interesting point. However, if that were the case, why would ELBs be restricted to single AWS regions? Anyway, its a nice feature that it just works without me needing to set it up.
- rbranson 16y ago
  Simple: distribute your application across two isolated datacenters. I'm not trying to downplay the complexity of doing this, as I fully understand the amount of effort required. If the cost of being down for an hour or two isn't significant enough to warrant this level of effort, then a power outage caused by Linode's datacenter vendor probably isn't going to really cost you much either.
- 16s 16y ago
  DNS
ahyes 16y ago
Could you provide an always-reliable example? Even anycast has it's downfalls (rogue networks advertising blocks & rerouting traffic onto their networks, security disaster). Most clustered setups rely on a central storage, and if your storage box loses connectivity, your site will be down. Additionally, if you're going to recommend mirroring san content to an identical box at a third party datacenter, the cost of transferring that is going to cost you a boatload of money in bandwidth bills, every single month.

KaeseEs 16y ago

Care to enumerate a few of them?

al_james 16y ago
More than one server across multiple datacenters is the only way. The use either a load balancer (which brings its own problems) or some kind of status aware DNS scheme to route traffic to live servers. I wish linode would offer some kind of high availability load balancing like amazon ec2 does, that really is a killer feature.
- rbranson 16y ago
  I wouldn't be so sure that ELB's are actually failing whole IP addresses across availability zones. Given the current ELB architecture is heavily dependent on dynamic DNS resolution, I am pretty suspicious that they are using DNS for failover. The separate set of DNS servers specifically for elb.amazonaws.com and the 60 second TTLs on the CNAMEs they generate seem to indicate this. Of course, there is no documentation to either confirm, or deny this. Amazon simply states in vague terms that it's highly available, but doesn't state how it's designed, or even possible failure outcomes.
  - al_james 16y ago
    Hmmm. Interesting point. However, if that were the case, why would ELBs be restricted to single AWS regions? Anyway, its a nice feature that it just works without me needing to set it up.
rbranson 16y ago
Simple: distribute your application across two isolated datacenters. I'm not trying to downplay the complexity of doing this, as I fully understand the amount of effort required. If the cost of being down for an hour or two isn't significant enough to warrant this level of effort, then a power outage caused by Linode's datacenter vendor probably isn't going to really cost you much either.
16s 16y ago
DNS

ahyes 16y ago

Could you provide an always-reliable example? Even anycast has it's downfalls (rogue networks advertising blocks & rerouting traffic onto their networks, security disaster). Most clustered setups rely on a central storage, and if your storage box loses connectivity, your site will be down. Additionally, if you're going to recommend mirroring san content to an identical box at a third party datacenter, the cost of transferring that is going to cost you a boatload of money in bandwidth bills, every single month.