3 ms·
S3 is designed for 11 9s of durability. What this means is that they model a "correctly functioning system" (i.e. replication and/or erasure coding) and the dur
by vlovich123 3y ago
S3 is designed for 11 9s of durability. What this means is that they model a "correctly functioning system" (i.e. replication and/or erasure coding) and the durability guarantee is about HW failures only. However, such models do not (& really cannot) account for the existence of bugs or introduction of new ones. That's a huge part of why S3 doesn't really do a whole lot of feature development (well that + it's hard to maintain a 20 yold codebase).
Also, we're talking about Google Drive here which isn't GCS (Google's S3 competitor) but a higher-level product layered on top of GCS but with it's own book keeping / ACLs etc. Thus there's more room for error. My hunch is that the data is permanently lost.
Additionally, S3 stores an enormous amount of data such that probabilistically they're bound to lose something to HW failure. 2 years ago, S3 stored 100 trillion objects [1]. With 11 9s of durability annually you'd expect to lose 100 objects a year. The saving grace is that most objects aren't accessed (maybe not ever again) & they detect & correct durability errors on access to ensure that accessed objects definitely aren't lost. So while they "admit" to 4 objects, that's likely an under count because I wouldn't expect them to regularly check if all 100 trillion objects are accessible because of how long that would take.
[1] https://www.zdnet.com/article/aws-s3-storage-now-holds-over-100-trillion-objects/ https://www.zdnet.com/article/aws-s3-storage-now-holds-over-...