4 ms·
> Programmers (myself included) tend to rather drastically underestimate the likelihood that data can end up in an error state. Have you priced ECC DDR4 recent
by m463 6y ago
> Programmers (myself included) tend to rather drastically underestimate the likelihood that data can end up in an error state.
Have you priced ECC DDR4 recently? That stuff is expensive.
What I find interesting is that the chances of a bit error is minute... but the number of bits in a common system is large. (Actually to a human mind I would say it is astronomically inconceivably large)
I first became aware that bit errors <=> memory size were colliding when I heard about large sun systems were doing something called memory scrubbing.
https://en.wikipedia.org/wiki/Memory_scrubbing https://en.wikipedia.org/wiki/Memory_scrubbing
- GauntletWizard 6y agoYeah. ZFS on modern systems schedules a complete read and “resilver” once a month. This is an aggressive schedule - But only “Probably”. I’m of the opinion that if you don’t have the capability of rereading 100% and rewriting 1% of your entire “cold” dataset every week, your data is already lost.
- m463 6y agoSo is this because flash is not as permanent as spinning disks? or is it memory related?
- GauntletWizard 6y agoFlash’s failure mode is generally better than spinning disk, in that complete drive failure (from head crash) is rarer and bad sectors, while technically “more common” on flash, are immensely predictable (X writes, +/- a few %) whereas on hard drives they’re fairly random. The real trick is that the forced exercise catches data errors (1 copy is missing/bad) long before they become corruptions (multiple copies are missing/bad). The bandwidth to read all of your flash disk in the space of a week is also much easier to come by, because flash has a much “better” ratio of bandwidth:storage size. The advice is very much rooted in being a “one size fits all” for both spinning rush and solid state - Spinning rust needs the checksums, flash can simply afford the waste.