3 ms·
You'd know much more about these sorts of things than I would, but most filesystems largely ignore disk corruption. If amazon did that, we'd be seeing a lot mo
by lsc 16y ago
You'd know much more about these sorts of things than I would, but most filesystems largely ignore disk corruption. If amazon did that, we'd be seeing a lot more trouble than we have. This is where I'm seeing the complexity; detecting and recovering from subtle errors. I've gone a short way down that path and from where I stand it looks quite hairy.
Besides, I've seen data loss caused by ext3 on my own systems that was not caused by disk corruption, so really, at s3's scale, it's got to function (and seems to have been functioning) much better than a regular filesystem.
- cperciva 16y agoS3 has an advantage over regular filesystems there too. S3 doesn't need to provide single-node durability as long as data loss events on different nodes are uncorrelated; so they can protect data with cryptographic checksums and throw it away (from an individual node) at the first sign of corruption, knowing that they'll still have it on all the other nodes.