4 ms·
What's the cloud solution (at major providers) that automatically geographically replicates data? For example, S3 buckets are tied to a region, which cannot be
by H8crilA 5y ago
What's the cloud solution (at major providers) that automatically geographically replicates data? For example, S3 buckets are tied to a region, which cannot be considered reliable - a single DC can always burn down, or even just intermittently unreachable. I'm looking for something that accepts an "upload" command that will (eventually) replicate the data to several regions. Ideally the regions can be changed at any point later, too.
This would take care of 3 and 1, sadly not 2 but 2 is pretty hard.
- gnur 5y agoS3 is region based, not AZ based. So it should be able to tolerate an entire AZ (=DC) being lost. They offer an amazing number of 9s for durability, it is very unlikely that a single fire will cause harm to your data.
- H8crilA 5y agoYeah, OK. But apart from the reliability question - what's a one stop shop for uploading files to Europe and the US in one go?
- seized 5y agoFor AWS there are options but require setup. It's not exactly one click setup and can be quite complex depending on your skill level. https://docs.aws.amazon.com/AmazonS3/latest/userguide/replication.html#crr-scenario https://docs.aws.amazon.com/AmazonS3/latest/userguide/replic... https://aws.amazon.com/blogs/aws/new-cross-region-replication-for-amazon-s3/ https://aws.amazon.com/blogs/aws/new-cross-region-replicatio...
- Heliosmaster 5y agoI can't find the article anymore, but a few weeks ago here on HN there was a piece where somebody did a few calculations on how AWS US-East-1 (the most used AWS region in the world) works, and got to the conclusion that all the things that can make that region go BOOM are likely to have major consequences to our civilization. In short: AWS will go down only when you won't care about it anyway
- ComputerGuru 5y agoS3 offers multi-region buckets that automatically read/write from one of n buckets in n regions. Cross-region replication rules give you eventual consistency (without conflict resolution) across all buckets, which is good enough for backup systems with unique file names/paths.
- H8crilA 5y agoThanks, this works! I'm going to use it now :)
- ComputerGuru 5y agoNo problem. I was loose with my lingo earlier; it’s not called a multi-region bucket but rather a multi-region S3 access point (hence the behavior described). More here: https://aws.amazon.com/s3/features/multi-region-access-points/ https://aws.amazon.com/s3/features/multi-region-access-point...
- seized 5y agoA region for AWS or GCP (and surely Azure) is more than one physical data center. A given data center (availability zone) can fail with no data loss. You have to choose very specific storage tiers with warnings galore to get to a point where one data center will cause data loss. There is also multi region buckets which go across regions. I use RClone and Duplicacy to backup to S3 Glacier and Google Workspace. Kopia is also another up and comer. For a simple copy type setup RClone is a good one to look at, by default it copies the files and doesn't do any snapshots, packing, splitting, etc.