4 ms·
There was an interesting quote/anecdote, Joe Armstrong likes to tell, it is about people who claim they've built a reliable or fault tolerant service. They woul
by rdtsc 13y ago
There was an interesting quote/anecdote, Joe Armstrong likes to tell, it is about people who claim they've built a reliable or fault tolerant service. They would say "This is fault tolerant, they are multiple hard drives in there, I have done formal verification of my code and so on..." and then someone else trips over the power cord and that's the end of the fault tolerance. It is just a silly example, of course they'd properly provide power to an important rack of hardware, but the point is, in the simplest case the system is only as fault tolerant as its weakest components. It is that one bad capacitor from Taiwan that might the whole thing down, or just a silly cosmic ray.
One needs redundant hardware to provide certain guarantees about the service being up. This means load balancers, multiple CPUs running the same code in parallel and comparing results, running on separate power buses, different data centers, different parts of the world.
- shurcooL 13y ago> different parts of the world. Still takes just one asteroid.
- oconnor0 13y agoSeems like after an asteroid hitting the earth, "server fault tolerance" is the least of our worries.
- klodolph 13y agoYeah, but with the number of 9s you see you realize that asteroids are NOT taken into account. For example, Amazon advertises 99.999999999% durability for a given year for S3 objects. This is just stupid. An extinction-level event (asteroid, global thermonuclear war, black hole) could easily wipe out ALL data on S3. We know that mass extinctions have occurred about once every 100 million years. That means that if we expect a 10^-8 chance of a mass extinction event in a given year, Amazon would need a 99% chance of surviving a mass extinction in order to meet average durability ratings for S3. After a certain number of 9s you just have to smile, nod, and truncate the number.
- josephagoss 13y agoThey really advertise 99.999999999%? Isn't that 1 byte lost for every 100,000,000 Terabytes or something silly like that?
- brianpgordon 13y agoYes: https://aws.amazon.com/s3/faqs/#How_durable_is_Amazon_S3 https://aws.amazon.com/s3/faqs/#How_durable_is_Amazon_S3 I suppose that if they ever lose an object then they can say "well we warned you that you might lose an object every hundred million years."
- mercurial 13y ago> That means that if we expect a 10^-8 chance of a mass extinction event in a given year, Amazon would need a 99% chance of surviving a mass extinction in order to meet average durability ratings for S3. On the other hand, it's unlikely that many people would be around to ask for a reclamation. It's more likely that Amazon would go out of business before anything like this happened, anyway.
- shurcooL 13y agoYeah, I was kinda referring to danger we as a species still face by being on just one planet.
- raverbashing 13y agoOff-topic: At one of my last jobs someone put the following ticket in the bug tracker "Following the end of the world on 12/21/2012 the system Blah will stop working" After the date it was closed with a "As the end of the world didn't happen this ticket is no longer needed"