3 ms·
"an object" Shouldn't that be multiplied by the number of bytes of this object? I still don't quite understand the math apparently because then we'd all be los
by Aachen 4y ago
"an object"
Shouldn't that be multiplied by the number of bytes of this object? I still don't quite understand the math apparently because then we'd all be losing 10KB+ files yearly.
- mbreese 4y agoI don’t think so. The question is: for any given file, what is the likelihood that if I restore it, there will be some corruption? (Total loss, bitrot, anything) For the purposes of this calculation, you would probably use the average size of a stored file, determine the number of chunks that was split into, and then the likelihood of losing/corrupting a chunk. By looking at the durability of objects, you (a) reinforce the object-store nature of S3, and (b) avoid reporting “durability per kb”, which isn’t linear, and more difficult to interpret. Honestly, the durability number is so high, that adding the bytes as a unit would just make it difficult to reason about. And S3 doesn’t store “bytes” — it stores objects (made of bytes). So, that’s the metric they chose to focus on.
- xani_ 4y agoIf we assume for the sake of argument simple RAID 6 setup, to lose say a block of data you'd need to * have a bad block on one drive * have a bad block in *EXACTLY SAME LOCATION* on the second drive * have a bad block in *EXACTLY SAME LOCATION* on the third drive * all of that happening within one scrub period (which is usually once a month). And that's just simple RAID6 setup without external redundancy. As you can see it would be far easier to lose data due to just correlated drive failures, than from random block corruption.