3 ms·
> 1 Okay fine, S3 too, but that’s a different animal. I think people forget that AWS S3 isn't immutable. Unlike an EBS volume, it is impossible to "snapshot" S
by 0xbadcafebee 5y ago
> 1 Okay fine, S3 too, but that’s a different animal.
I think people forget that AWS S3 isn't immutable. Unlike an EBS volume, it is impossible to "snapshot" S3 the way you can a database. There are arbitrary global limitations outside the scope of your control, and a dozen different problems with trying to restore or move or version all the things about buckets that aren't objects (although the objects too can be problematic depending on a series of factors).
If you want real simplicity/repeatability/reliability, but have to use S3, host your own internal S3 service. This way you can completely snapshot both the metadata and block devices used by your S3 service, making it actually immutable. Plus you can do things like re-use a bucket name or move it to any region without worrying about global conflicts. (All of that is hard/expensive to do, however, so you should just use AWS S3 and be very careful not to use it in a way that is unreliable)
- electroly 5y ago> it is impossible to "snapshot" S3 the way you can a database FWIW, they added point-in-time S3 snapshots via AWS Backup recently.
- 0xbadcafebee 5y agoHave you seen how complicated it is? https://aws.amazon.com/blogs/storage/point-in-time-restore-for-amazon-s3-buckets/ https://aws.amazon.com/blogs/storage/point-in-time-restore-f... It also seems to only be object changes, not all bucket changes, so treating the entire bucket as an immutable versioned artifact still isn't possible.
- deleted 5y ago[deleted]
- simonw 5y agoMy guess is that they use S3 mainly for things like backups, where you write once to a brand new key. I'd be surprised if they were using mutable S3 objects that constantly get updated in-place. They have PostgreSQL for that!
- deleted 5y ago[deleted]