4 ms·
This is a great lesson in how not to be a pedantic nerd toward clients. By 100%, the client is obviously not specifying an uptime requirement of: lim (1
by biot 13y ago
This is a great lesson in how not to be a pedantic nerd toward clients. By 100%, the client is obviously not specifying an uptime requirement of:
lim (1 - 1/x)
x->inf
They're saying that downtime is unacceptable. As the consultant/contractor, it's your job to present various scenarios and reiterate to the client that redundancy/failover features introduced at each new level roughly doubles the hosting cost plus requires additional engineering time for all the planning, prototyping, development, testing, deployment, and so on for the sync/failover functionality. For example (successive levels include redundancies of previous levels):
Level 1: Single server is connected to a UPS device. Domain's DNS records from multiple DNS hosting companies.
Level 2: Redundant servers load balanced on the firewall.
Level 3: Pool of redundant servers with redundant firewalls and redundant network switches.
Level 4: Geographically separated data centers with traffic routed via anycast.
Level 5: Servers in multiple datacenters from multiple distinct vendors.
Outline ballpark costs at each level plus introduce the required staffing levels to support each level including site reliability engineers to monitor and maintain each site's operation and verify correct data synchronization, 24x7 on-call engineers, equipment to enable remote diagnostics, and so on.
- 300bps 13y ago>This is a great lesson in how not to be a pedantic nerd toward clients I've been in IT for 20 years so I may be jaded, but I disagree with your conclusion. Your Levels don't seem to scratch the surface on achieving even 99.999% uptime. First of all, if you are planning on using vendors for your hypothetical 99.999% uptime you can stop there. Sure, they may promise that to you. But what is the penalty if they don't? If you are planning on hosting this in your own data centers... Are you including diesel generators and massive diesel tanks to supply them in the event of extended power outages? What is your DDoS plan? What is your plan in the event of a hacker? What's your patching plan? These are just examples. The level of planning to achieve 99.999% uptime per year is massive. That's only 5 minutes of downtime per year!! The bottom line is there are a handful of applications in the world that require this level of uptime. Unfortunately, every single client I have ever had was under the impression they required this level of uptime. It is my job to explain to them what reality is. If I don't, then I am not allowing them to prepare for the inevitable times that the system will be down no matter how much they spend or what they do.
- biot 13y agoI totally agree. I was assuming that diesel backup generators with refuelling contracts in place is considered table stakes for a data center these days. My example levels weren't exhaustive by any means and your points around capacity planning, attack mitigation, change management, etc. are spot on.
- twistedpair 13y agoDon't forget to plan for when your datacenter explodes and the building is condemned. http://www.datacenterknowledge.com/archives/2008/06/01/explosion-at-the-planet-causes-major-outage/ http://www.datacenterknowledge.com/archives/2008/06/01/explo...