3 ms·

Five way redundancy look good on paper, put if you use the same vendor and hardware purchased at the same time, you have a one point of failure, aka when one se

by z3t4 10y ago

Five way redundancy look good on paper, put if you use the same vendor and hardware purchased at the same time, you have a one point of failure, aka when one server fails there's a high probability that the others will fail simultaneously.

davnn 10y ago
Why?
- smallnamespace 10y ago
  If the hardware failure is due to a manufacturing problem, then it affects every machine. If it's problem with the production batch, then again, unless you bought from different batches, you haven't reduced risk much. And if it's a software problem, then again the redundancy isn't helping you. Basically you haven't succeeded in getting 5 independent tries in the problem -- the failures are highly correlated. It's the same reasoning that led to the financial crisis -- you have a CDO or an MBS that has a bunch of different obligations so you think you've diversified away risk, but in actuality there is one big factor that causes everything to fail at once.
  - throwawayish 10y ago
    Also - same shipping way. If I buy four disks from the same shop, then those four disks came in the same delivery to the shop (so they were all dropped at the same time), are handled by the same picker, thrown into the same parcel and dropped a couple times the same way.
eeZah7Ux 10y ago
By running statistics on a large enough infrastructure you can see how frequently 2 hard drives in the same raid fail within days, or even hours! Same brand, same batch, operating time, running temperature, even same vibration does trick. :(
- chiph 10y ago
  We had 3 hard drives (the legendary IBM DeathStars) from the same production batch fail within hours of each other. Part of me was like "Yeah! Awesome quality control!" and a larger part was like "Oh shit." We ended up losing the array and had to restore from backup.
sllabres 10y ago
It depends on what you are trying to protect and what options you have if one component fails (replacement). For highly critical systems that is an issue but it is being adressed. Take as example the space shuttle or the Airbus (3/2) primary/secondary flight computer [1] where the backup is being build by different companies with different processors ... [1] https://ifs.host.cs.st-andrews.ac.uk/Resources/CaseStudies/Airbus/Airbus-fcs.pdf https://ifs.host.cs.st-andrews.ac.uk/Resources/CaseStudies/A...

by z3t4 10y ago

davnn 10y ago

Why?

smallnamespace 10y ago
If the hardware failure is due to a manufacturing problem, then it affects every machine. If it's problem with the production batch, then again, unless you bought from different batches, you haven't reduced risk much. And if it's a software problem, then again the redundancy isn't helping you. Basically you haven't succeeded in getting 5 independent tries in the problem -- the failures are highly correlated. It's the same reasoning that led to the financial crisis -- you have a CDO or an MBS that has a bunch of different obligations so you think you've diversified away risk, but in actuality there is one big factor that causes everything to fail at once.
- throwawayish 10y ago
  Also - same shipping way. If I buy four disks from the same shop, then those four disks came in the same delivery to the shop (so they were all dropped at the same time), are handled by the same picker, thrown into the same parcel and dropped a couple times the same way.

eeZah7Ux 10y ago

By running statistics on a large enough infrastructure you can see how frequently 2 hard drives in the same raid fail within days, or even hours! Same brand, same batch, operating time, running temperature, even same vibration does trick. :(

chiph 10y ago
We had 3 hard drives (the legendary IBM DeathStars) from the same production batch fail within hours of each other. Part of me was like "Yeah! Awesome quality control!" and a larger part was like "Oh shit." We ended up losing the array and had to restore from backup.

sllabres 10y ago

It depends on what you are trying to protect and what options you have if one component fails (replacement). For highly critical systems that is an issue but it is being adressed. Take as example the space shuttle or the Airbus (3/2) primary/secondary flight computer [1] where the backup is being build by different companies with different processors ... [1] https://ifs.host.cs.st-andrews.ac.uk/Resources/CaseStudies/Airbus/Airbus-fcs.pdf https://ifs.host.cs.st-andrews.ac.uk/Resources/CaseStudies/A...